-- Logs begin at Mon 2022-05-23 16:19:06 UTC, end at Mon 2022-05-23 17:10:27 UTC. -- May 23 16:52:30.958525 dv-cinder-842541-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 23 16:52:30.967539 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 23 16:52:30.969561 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon May 23 16:52:30 2022] *** May 23 16:52:30.969561 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 23 16:52:30.969561 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 23 16:52:30.969561 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: nodename: dv-cinder-842541-2-iscsi May 23 16:52:30.969561 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: machine: x86_64 May 23 16:52:30.969561 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: clock source: unix May 23 16:52:30.969561 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: pcre jit disabled May 23 16:52:30.969561 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: detected number of CPU cores: 4 May 23 16:52:30.969561 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: current working directory: / May 23 16:52:30.969561 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: detected binary path: /usr/bin/uwsgi-core May 23 16:52:30.970074 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: your processes number limit is 47680 May 23 16:52:30.970074 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: your memory page size is 4096 bytes May 23 16:52:30.970074 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: detected max file descriptor number: 2048 May 23 16:52:30.970074 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: lock engine: pthread robust mutexes May 23 16:52:30.970074 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: thunder lock: enabled May 23 16:52:30.970074 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 23 16:52:30.970074 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 23 16:52:31.014161 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: Python main interpreter initialized at 0x5581067cd9c0 May 23 16:52:31.014161 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: python threads support enabled May 23 16:52:31.014161 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: your server socket listen backlog is limited to 100 connections May 23 16:52:31.014161 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: your mercy for graceful operations on workers is 90 seconds May 23 16:52:31.014511 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: mapped 403077 bytes (393 KB) for 2 cores May 23 16:52:31.014587 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: *** Operational MODE: preforking *** May 23 16:52:31.014835 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: *** uWSGI is running in multiple interpreter mode *** May 23 16:52:31.014835 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: spawned uWSGI master process (pid: 80457) May 23 16:52:31.015026 dv-cinder-842541-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 23 16:52:31.018945 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: spawned uWSGI worker 1 (pid: 80464, cores: 1) May 23 16:52:31.018945 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: spawned uWSGI worker 2 (pid: 80465, cores: 1) May 23 16:52:31.018945 dv-cinder-842541-2-iscsi devstack@keystone.service[80457]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 23 16:52:36.585190 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 16:52:36.585190 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: from cryptography.utils import int_from_bytes May 23 16:52:36.585982 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 16:52:36.585982 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: from cryptography.utils import int_from_bytes May 23 16:52:36.586389 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 16:52:36.586389 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: from cryptography.utils import int_from_bytes May 23 16:52:36.586495 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 16:52:36.586495 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: from cryptography.utils import int_from_bytes May 23 16:52:36.634202 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 23 16:52:36.634406 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 23 16:52:36.634538 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] command line args: [] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 23 16:52:36.634822 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 23 16:52:36.635305 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 23 16:52:36.635395 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ================================================================================ {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 23 16:52:36.635463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 23 16:52:36.635593 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] admin_token = **** {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.635676 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] command line args: [] {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 23 16:52:36.635827 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] config_dir = [] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.635958 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 23 16:52:36.636048 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.636155 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] config_source = [] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.636246 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ================================================================================ {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 23 16:52:36.636324 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] debug = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.636607 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.638087 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] admin_token = **** {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.638087 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] config_dir = [] {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.638219 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.638285 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.638348 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.638406 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] config_source = [] {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.638406 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] debug = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.638524 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.638583 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.638583 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.638583 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.638777 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.638841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.638903 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] list_limit = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.638992 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.639049 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] log_config_append = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.639118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] list_limit = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.639190 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.639250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] log_config_append = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.639306 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] log_dir = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.639363 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.639418 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] log_file = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.639481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] log_dir = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.639538 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.639594 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] log_file = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.639813 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.639895 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.640032 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.640171 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.640250 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.640330 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.640464 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.640547 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.640620 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.640749 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.640835 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.640912 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.641033 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.641106 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.641232 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.641312 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.641483 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.641613 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.641693 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.641764 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.641922 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.642041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.642113 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.642196 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.642370 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.642492 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.642586 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.642667 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.642871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.643151 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.643285 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] publish_errors = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.643374 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.13/identity {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.643536 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] publish_errors = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.643658 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.643758 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.643889 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.644003 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.644113 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.644113 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.644318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.644490 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.644613 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.644725 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.644841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] standard_threads = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.644952 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] standard_threads = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.645069 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.645187 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.645304 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.645304 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.645512 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.645621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] use_journal = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.645741 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.645852 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] use_json = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.645960 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] use_journal = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.646069 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] use_stderr = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.646179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] use_json = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.646287 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] use_syslog = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.646404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] use_stderr = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.646404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] use_syslog = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.646612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.646721 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 16:52:36.646836 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.646969 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.647098 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.647211 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.647321 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.647430 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.647546 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.647668 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.647786 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.647903 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.648014 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.648124 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.648239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.648432 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.648545 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.648668 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.648780 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.648889 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.648999 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.649117 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.649228 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.649337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.649446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.649555 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.649672 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.649790 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.649902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.650016 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.650133 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.650244 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.650354 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.650472 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.650583 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.650694 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.650803 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.650927 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.651038 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.651164 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.651277 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.651387 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.651505 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.651620 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.651731 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.651853 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.651955 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.652081 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.652193 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.652309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.652404 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.652504 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.652600 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.652699 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.652816 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.652909 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.653001 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.653001 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.653182 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.653295 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.653391 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.653483 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.653594 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.653781 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.653896 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.653896 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.654114 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.654256 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.654256 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.654474 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.654590 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.654706 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.654845 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.654971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.655096 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.655209 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.655319 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.655427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.655537 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.655652 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.655770 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.655881 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.655997 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.656113 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.656224 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.656332 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.656449 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.656567 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.656676 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.656875 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.657092 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.657220 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.657417 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.657629 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.657946 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.658151 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.658280 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.658384 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.658494 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.658704 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.658898 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.659294 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.659511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.659668 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.659882 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.660030 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.660269 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.660443 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.660591 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.660758 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.660931 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.661095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.661242 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.661242 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.661532 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.661698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.661698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.661995 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.662171 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.662420 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.662588 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.662754 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.662945 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.663296 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.663441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.663618 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.663783 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.663946 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.664085 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.664236 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.664383 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.664497 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.664633 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.664755 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.664868 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.665006 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.665129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.665240 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.665352 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.665462 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.665578 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.665716 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.665716 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.665949 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.666071 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.666071 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.666279 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.666396 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.666516 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.666634 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.666634 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.666841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.666969 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.666969 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.667228 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.667346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.667465 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.667577 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.667687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.667876 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.667994 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.668106 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.668216 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.668325 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.668435 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.668555 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.668667 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.668777 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.668887 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.668997 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.669144 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.669271 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.669384 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.669509 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.669630 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.669772 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.669913 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.669913 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.670138 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.670138 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.670346 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.670455 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.670455 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.670671 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.670791 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.670931 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.671046 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.671157 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.671273 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.671385 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.671495 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.671495 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.671729 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.671841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.671841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.672048 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.672158 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.672158 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] auth.external = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.672384 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.672496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.672606 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.672718 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.672829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.672947 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.673059 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.673176 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] auth.password = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.673288 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.673398 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] auth.token = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.673508 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.673627 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.673740 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.673858 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.673976 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.674087 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.674205 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.674347 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.674471 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.674594 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.674714 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.674827 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.675046 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.675162 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.675273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.675397 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] credential.caching = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.675511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] auth.external = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.675632 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.675744 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.675856 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.675976 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.676098 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.676212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.676323 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.676435 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] auth.password = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.676546 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.676665 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] auth.token = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.676779 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.676891 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.677003 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.677115 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.677225 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.677344 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.677459 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.677571 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.677683 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.677794 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.677954 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.678077 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.678277 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.678394 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.678394 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] credential.caching = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.678621 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.678735 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.678854 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.678984 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.679097 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.679209 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.679332 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] federation.caching = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.679445 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.679570 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.679684 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.679796 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.679914 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.680027 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.680146 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.680266 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.680379 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.680492 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.680726 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.680839 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.680950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.681073 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.681189 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.681307 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.681426 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.681539 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.681650 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.681840 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.681955 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.682067 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.682180 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.682291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] identity.caching = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.682403 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] federation.caching = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.682529 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.682643 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.682761 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.682874 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.683036 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.683150 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.683277 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.683391 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.683511 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.683623 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.683735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.683846 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.683966 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.684079 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.684193 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.684306 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.684417 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.684528 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.684648 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.684797 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.684943 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.685055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.685166 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.685277 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.caching = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.685397 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.685510 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.685621 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.685739 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.685852 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.685972 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.686164 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.686286 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.686399 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.686511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.686622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.686742 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.686855 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.686982 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.687094 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.687206 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.687318 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.687444 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.687558 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.687670 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.687783 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.687895 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.688007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.688127 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.688241 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.688360 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.688474 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.688586 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.688698 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.688817 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.688930 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.689042 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.689153 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.689265 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.689376 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.689376 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.689596 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.689707 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.689818 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.689929 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.690040 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.690159 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.690272 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.690384 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.690495 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.690607 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.690724 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.690845 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.690983 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.691097 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.691209 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.691321 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.691432 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.691556 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.691671 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.691783 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.691894 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.692004 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.692106 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.692229 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.692343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.692343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.692554 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.692673 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.692785 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.692904 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.693017 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.693127 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.693238 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.693349 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.693467 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.693619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.693734 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.693845 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.693955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.694066 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.694176 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.user = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.694176 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.694394 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.694505 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.694622 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.694735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.694846 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.694983 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.695097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.695208 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.695208 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.695406 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.695516 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.695639 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.695754 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.695865 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.695976 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.695976 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.696185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.696304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.696417 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.696528 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.696638 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.696749 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.696861 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.696981 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.697094 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.697205 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.697316 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.697427 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.697547 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.697660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.697771 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.697883 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.697994 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.698104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.698223 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.698336 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.698447 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.698557 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.698668 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.698779 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.698897 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.699044 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.699158 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.699270 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.699381 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.699492 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.699612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.699725 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.699837 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.699948 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.700060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.700171 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.700291 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.700404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.700515 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.700627 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.700737 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.700848 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.700968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.701082 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.701194 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.701304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.701416 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.701527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.701661 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.701829 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.701944 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.702056 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.702167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.702278 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] resource.caching = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.702401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.702401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.702611 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.702722 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.702833 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.703054 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.703184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.703299 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.703410 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.703524 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.703643 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.703756 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.703938 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.704054 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.704166 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.704278 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.704390 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.704532 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.704655 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.704769 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.704880 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] role.caching = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.704992 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.705103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] role.driver = sql {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.705215 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.705335 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.705449 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.705560 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.705671 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.705782 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.705894 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] resource.caching = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.706013 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.706013 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.706224 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.706336 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.706447 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.706558 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.706677 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.706795 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.706931 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.707046 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.707157 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.707267 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.707392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.707506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.707617 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.707728 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.707838 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.707949 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.708069 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.708182 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.708293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] role.caching = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.708293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] role.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.708503 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.708622 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.708734 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.708847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.708958 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.709070 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.709181 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.709300 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.709415 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.709527 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.709638 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.709749 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.709862 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.709981 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.710095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.710206 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.710317 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.710317 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.710527 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.710647 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.710759 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.710871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.710996 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.711108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.711220 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.711343 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.711457 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.711570 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.711682 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.711792 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.711792 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.712012 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.712125 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.712240 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.712352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.712464 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.712577 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] token.caching = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.712696 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.712799 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.712909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.713020 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.713167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.713281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.713391 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.713504 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.713616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.713727 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.713837 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.713948 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.714067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.714179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.714291 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.714402 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.714512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.714631 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.714745 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.714856 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.715016 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.715129 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.715240 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.715363 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.715477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.715589 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.715700 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.715814 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.715931 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.716052 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.716165 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.716277 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.716388 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] token.caching = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.716498 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.716610 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.716728 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] database.connection = **** {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.716852 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.716977 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.717091 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.717201 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.717312 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.717423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.717534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.717653 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.717767 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.717878 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.717990 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.718101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.718212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.718331 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.718480 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.718591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.718702 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.718813 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.718949 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.719075 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.719189 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.719301 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.719412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.719523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.719635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.719755 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.719868 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.720001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.720113 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.720224 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.720224 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.720446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.720593 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.720707 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.connection = **** {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.720828 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.721004 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.721121 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80465) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 23 16:52:36.721233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.721233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.721233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.721233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.721233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.721233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.721233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.722016 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.722016 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.722016 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.722016 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.722459 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.722459 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.722677 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.722865 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.723128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.723358 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 16:52:36.723524 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80464) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 23 16:52:36.750843 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.751028 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.751242 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.753357 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.753591 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.753817 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.760357 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.763945 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.790709 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.790897 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.796899 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.797136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.844663 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.850722 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.903522 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.903737 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.912535 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.912782 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.980256 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.980538 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.980632 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.980760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.980942 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.981086 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.981211 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.981333 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.981455 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.981591 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.981719 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:36.981847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.008712 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.008967 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.009123 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.009282 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.009500 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.009709 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.010243 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.010747 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.010968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.011185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.011430 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.011737 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.011929 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.012168 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.024269 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.026931 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.039079 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.039323 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.039487 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.039579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.039703 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.039824 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.039945 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.040074 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.040236 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.040373 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.040496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80465) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.040618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.040618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.040867 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80464) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 16:52:37.061971 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.062160 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 16:52:37.062490 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.062682 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 16:52:37.062933 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.063227 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.063448 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.063729 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.065165 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.065363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 16:52:37.065703 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.065898 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 16:52:37.066140 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.066528 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.066799 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.067084 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.079274 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.079527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.079816 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.080096 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.080725 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.081392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.081667 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.081947 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.088631 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80465) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.091040 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.092018 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80465) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.092290 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.092565 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.092838 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.093171 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.093404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.093695 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.094070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.094336 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.094742 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.094861 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.094962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.095230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.095495 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.095771 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.096051 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.096310 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.096577 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.096847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.097110 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.097475 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.097742 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.129168 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.130901 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80465) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.131351 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.133135 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.134774 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.135397 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.137156 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80465) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.137433 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.137705 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.140998 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.141281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.141552 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.145182 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80465) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.145461 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.149533 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.149824 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.153620 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80465) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.153906 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.154184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.154672 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.154946 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.155094 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.155223 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.155361 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.155486 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.155748 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.155944 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.156311 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.156656 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.157051 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.157324 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.157783 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.174841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.175116 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.182693 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.183092 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.183408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80465) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.187149 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.187441 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.188364 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.192203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.192499 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.192624 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80465) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.196360 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80465) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.197959 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.202022 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.203474 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.203769 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.204174 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.204460 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.204747 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.205030 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.206397 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.206933 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.207424 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.207895 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.208183 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.208475 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.224878 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80465) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.225265 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.225532 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.225792 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.232567 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.232857 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.233125 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.233387 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.234158 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.235873 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.236145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.239208 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80465) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.239479 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.239755 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.240029 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.240299 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.241796 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.244121 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.244452 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.247555 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.247832 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.248108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.248384 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.248655 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.254155 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80465) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.254441 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.254708 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.254971 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.255274 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.255548 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.255824 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.265689 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.265872 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.266172 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.266464 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.266741 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.267079 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.267555 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.278776 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80465) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.282186 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80465) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.285883 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.287610 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.289325 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80465) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.289609 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.289883 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.293249 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.296728 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.300678 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.301209 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.302897 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.303088 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80465) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.305701 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.306155 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.306348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.306607 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.306734 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.306893 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.307196 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.316524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80465) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.316748 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.316849 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.317063 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.318500 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.322712 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.323024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.323312 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.323581 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.329020 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80465) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.329299 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.329575 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.329850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.330114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.330383 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.330668 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.330926 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.331233 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.331493 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.331753 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.331979 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.332257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.332506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80465) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.334822 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.335209 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.335640 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.344015 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80464) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 16:52:37.344301 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.344755 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.345168 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.345483 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.345757 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.346042 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.346316 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.346738 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.347077 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.347349 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.347614 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.348062 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.348451 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80464) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 16:52:37.381792 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5581067cd9c0 pid: 80465 (default app) May 23 16:52:37.385626 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821eb5a7-a72f-4bee-968a-5d8c600ccc43 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:37.385913 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821eb5a7-a72f-4bee-968a-5d8c600ccc43 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:37.386186 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-821eb5a7-a72f-4bee-968a-5d8c600ccc43 None None] PATH_INFO: `/v3` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:37.390485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 1/1] 10.222.0.13 () {50 vars in 791 bytes} [Mon May 23 16:52:37 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:52:37.397614 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5581067cd9c0 pid: 80464 (default app) May 23 16:52:45.326940 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435569d-cd47-4411-8aea-6efbc0ae0bff None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:45.327527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435569d-cd47-4411-8aea-6efbc0ae0bff None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:45.327527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6435569d-cd47-4411-8aea-6efbc0ae0bff None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:45.328831 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 2/2] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:52:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:52:45.336470 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d20d3-1432-45f0-80f9-ef8d692d5ac7 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:45.336906 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d20d3-1432-45f0-80f9-ef8d692d5ac7 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:45.336994 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582d20d3-1432-45f0-80f9-ef8d692d5ac7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:45.408919 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: INFO dbcounter [None req-582d20d3-1432-45f0-80f9-ef8d692d5ac7 None None] Registered counter for database keystone May 23 16:52:45.434764 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG oslo_db.sqlalchemy.engines [None req-582d20d3-1432-45f0-80f9-ef8d692d5ac7 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80464) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 16:52:45.446702 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writer thread running {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 16:52:45.629684 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG passlib.handlers.bcrypt [None req-582d20d3-1432-45f0-80f9-ef8d692d5ac7 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80464) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 23 16:52:45.630091 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG passlib.handlers.bcrypt [None req-582d20d3-1432-45f0-80f9-ef8d692d5ac7 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80464) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 23 16:52:45.688055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-582d20d3-1432-45f0-80f9-ef8d692d5ac7 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:52:45.689663 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-582d20d3-1432-45f0-80f9-ef8d692d5ac7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:52:45.817332 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 1/3] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:52:45 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 484 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 23 16:52:45.986891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6435569d-cd47-4411-8aea-6efbc0ae0bff None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:46.061289 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: INFO dbcounter [None req-6435569d-cd47-4411-8aea-6efbc0ae0bff None None] Registered counter for database keystone May 23 16:52:46.082192 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG oslo_db.sqlalchemy.engines [None req-6435569d-cd47-4411-8aea-6efbc0ae0bff None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80465) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 16:52:46.094325 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writer thread running {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 16:52:46.222311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6435569d-cd47-4411-8aea-6efbc0ae0bff None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:46.227937 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ac702c-1977-4668-9e59-51044031f862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841c10778a1544ddb64ccc3d96fecf42', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:46.228596 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac702c-1977-4668-9e59-51044031f862 admin admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:46.228880 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac702c-1977-4668-9e59-51044031f862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:46.229152 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ac702c-1977-4668-9e59-51044031f862 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:46.238328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac702c-1977-4668-9e59-51044031f862 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:46.295964 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: WARNING keystone.common.rbac_enforcer.enforcer [None req-16ac702c-1977-4668-9e59-51044031f862 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 23 16:52:46.296278 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ac702c-1977-4668-9e59-51044031f862 admin admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:46.307088 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 3/4] 10.222.0.13 () {58 vars in 1324 bytes} [Mon May 23 16:52:45 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 16:52:46.313683 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d20d3-1432-45f0-80f9-ef8d692d5ac7 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:46.329308 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582d20d3-1432-45f0-80f9-ef8d692d5ac7 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:46.335012 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7547c45-1b39-40bf-9a26-c4bf71167ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841c10778a1544ddb64ccc3d96fecf42', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:46.336165 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7547c45-1b39-40bf-9a26-c4bf71167ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:46.336452 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7547c45-1b39-40bf-9a26-c4bf71167ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:46.336725 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7547c45-1b39-40bf-9a26-c4bf71167ac9 admin admin] PATH_INFO: `/v3/projects/841c10778a1544ddb64ccc3d96fecf42` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:46.338497 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7547c45-1b39-40bf-9a26-c4bf71167ac9 admin admin] RBAC: Authorizing `identity:get_project(project_id=841c10778a1544ddb64ccc3d96fecf42)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:46.498104 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: WARNING keystone.common.rbac_enforcer.enforcer [None req-b7547c45-1b39-40bf-9a26-c4bf71167ac9 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 23 16:52:46.498390 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7547c45-1b39-40bf-9a26-c4bf71167ac9 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:46.503153 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 2/5] 10.222.0.13 () {56 vars in 1207 bytes} [Mon May 23 16:52:46 2022] GET /identity/v3/projects/841c10778a1544ddb64ccc3d96fecf42 => generated 344 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:52:48.661935 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995945c-d81f-41c2-b10d-5ba7115366a1 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:48.662750 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995945c-d81f-41c2-b10d-5ba7115366a1 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:48.662863 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0995945c-d81f-41c2-b10d-5ba7115366a1 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:48.669625 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 4/6] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:52:48 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:52:48.675339 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfd918-411c-4f20-b6c7-8b97e7b4a53e None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:48.675761 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfd918-411c-4f20-b6c7-8b97e7b4a53e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:48.675851 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfd918-411c-4f20-b6c7-8b97e7b4a53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:48.696983 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-c8dfd918-411c-4f20-b6c7-8b97e7b4a53e None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:52:48.698499 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c8dfd918-411c-4f20-b6c7-8b97e7b4a53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:52:48.706958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 3/7] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:52:48 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 23 16:52:48.730157 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995945c-d81f-41c2-b10d-5ba7115366a1 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:48.752039 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0995945c-d81f-41c2-b10d-5ba7115366a1 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:48.759371 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131e0000-5e1c-47a9-8008-69de36c64f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841c10778a1544ddb64ccc3d96fecf42', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:48.760334 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e0000-5e1c-47a9-8008-69de36c64f1e admin admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:48.760619 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e0000-5e1c-47a9-8008-69de36c64f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:48.760889 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131e0000-5e1c-47a9-8008-69de36c64f1e admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:48.769469 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e0000-5e1c-47a9-8008-69de36c64f1e admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:48.771309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131e0000-5e1c-47a9-8008-69de36c64f1e admin admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:48.780652 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 5/8] 10.222.0.13 () {58 vars in 1324 bytes} [Mon May 23 16:52:48 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 16:52:48.786517 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dfd918-411c-4f20-b6c7-8b97e7b4a53e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:48.795890 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dfd918-411c-4f20-b6c7-8b97e7b4a53e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:48.800784 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f14ab90-d3df-4cea-a9dd-9766ef3b2940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841c10778a1544ddb64ccc3d96fecf42', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:48.801653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14ab90-d3df-4cea-a9dd-9766ef3b2940 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:48.801939 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14ab90-d3df-4cea-a9dd-9766ef3b2940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:48.802219 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f14ab90-d3df-4cea-a9dd-9766ef3b2940 admin admin] PATH_INFO: `/v3/users/d1134eaf564846f5990bd2b2934b7db7` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:48.803715 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14ab90-d3df-4cea-a9dd-9766ef3b2940 admin admin] RBAC: Authorizing `identity:get_user(user_id=d1134eaf564846f5990bd2b2934b7db7)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:48.805558 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f14ab90-d3df-4cea-a9dd-9766ef3b2940 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:48.807150 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 4/9] 10.222.0.13 () {56 vars in 1198 bytes} [Mon May 23 16:52:48 2022] GET /identity/v3/users/d1134eaf564846f5990bd2b2934b7db7 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:52:51.043019 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf751f31-58b5-4a9b-a4e5-56e247826fa8 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:51.043536 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf751f31-58b5-4a9b-a4e5-56e247826fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:51.043637 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf751f31-58b5-4a9b-a4e5-56e247826fa8 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:51.044914 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 6/10] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:52:51.050153 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c61033-a70b-4512-ad8a-a45379851829 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:51.050594 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c61033-a70b-4512-ad8a-a45379851829 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:51.050694 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c61033-a70b-4512-ad8a-a45379851829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:51.058302 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b5517-9a49-4464-89e1-ad4cd4c78d60 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:51.058592 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b5517-9a49-4464-89e1-ad4cd4c78d60 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:51.058872 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872b5517-9a49-4464-89e1-ad4cd4c78d60 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:51.060202 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 7/11] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:52:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:52:51.065241 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb845f-4e3a-4ef6-b41f-d6eab4b14e00 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:51.065527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb845f-4e3a-4ef6-b41f-d6eab4b14e00 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:51.065804 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bb845f-4e3a-4ef6-b41f-d6eab4b14e00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:51.068087 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-59c61033-a70b-4512-ad8a-a45379851829 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:52:51.069327 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-59c61033-a70b-4512-ad8a-a45379851829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:52:51.105944 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG passlib.handlers.bcrypt [None req-29bb845f-4e3a-4ef6-b41f-d6eab4b14e00 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80465) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 23 16:52:51.106374 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG passlib.handlers.bcrypt [None req-29bb845f-4e3a-4ef6-b41f-d6eab4b14e00 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80465) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 23 16:52:51.153069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-29bb845f-4e3a-4ef6-b41f-d6eab4b14e00 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:52:51.154678 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-29bb845f-4e3a-4ef6-b41f-d6eab4b14e00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:52:51.163230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 5/12] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:52:51.185777 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c61033-a70b-4512-ad8a-a45379851829 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:51.211828 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:52:51.255926 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 8/13] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 16:52:51.277724 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb845f-4e3a-4ef6-b41f-d6eab4b14e00 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:51.296733 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c61033-a70b-4512-ad8a-a45379851829 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:51.331546 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dabe42-eee6-446e-8bd3-1be0291b1a4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:51.332365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dabe42-eee6-446e-8bd3-1be0291b1a4e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:51.332647 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dabe42-eee6-446e-8bd3-1be0291b1a4e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:51.332921 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dabe42-eee6-446e-8bd3-1be0291b1a4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:51.346541 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89dabe42-eee6-446e-8bd3-1be0291b1a4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:52:51.358388 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-89dabe42-eee6-446e-8bd3-1be0291b1a4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:52:51.358388 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:52:51.358388 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:52:51.358388 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:52:51.358388 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:52:51.358388 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:52:51.358388 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:52:51.358388 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:52:51.358388 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:52:51.358388 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:52:51.358388 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:52:51.358388 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:52:51.358388 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:52:51.358388 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:52:51.358388 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:52:51.358388 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:52:51.359506 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:52:51.359506 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:52:51.359506 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:52:51.359506 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:52:51.359506 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:52:51.359506 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:52:51.359506 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:52:51.359506 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:52:51.359506 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:52:51.359506 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:52:51.359506 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:52:51.359506 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:52:51.359506 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:52:51.359506 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:52:51.359506 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:52:51.359506 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:52:51.360306 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:52:51.360306 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:52:51.360306 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:52:51.360306 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:52:51.360306 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:52:51.360306 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:52:51.360306 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:52:51.360306 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:52:51.360306 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:52:51.360306 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:52:51.360306 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:52:51.360306 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:52:51.360306 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 6/14] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:52:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:52:51.366509 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dabe42-eee6-446e-8bd3-1be0291b1a4e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:51.457136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:52:51.479808 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dabe42-eee6-446e-8bd3-1be0291b1a4e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:51.499580 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bb845f-4e3a-4ef6-b41f-d6eab4b14e00 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:51.502942 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:52:51.515793 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a044fe6-3355-46e5-b433-45891de5aec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:51.516479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a044fe6-3355-46e5-b433-45891de5aec6 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:51.516762 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a044fe6-3355-46e5-b433-45891de5aec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:51.517040 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a044fe6-3355-46e5-b433-45891de5aec6 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:51.517805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a044fe6-3355-46e5-b433-45891de5aec6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:52:51.518125 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a044fe6-3355-46e5-b433-45891de5aec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:51.519696 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a044fe6-3355-46e5-b433-45891de5aec6 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:51.531914 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 7/15] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:52:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:52:51.537453 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a044fe6-3355-46e5-b433-45891de5aec6 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:51.537764 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f5941-32ad-4bd2-912b-7c350eecf1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:51.538682 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f5941-32ad-4bd2-912b-7c350eecf1db None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:51.539064 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f5941-32ad-4bd2-912b-7c350eecf1db None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:51.539346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383f5941-32ad-4bd2-912b-7c350eecf1db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:51.609209 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f5941-32ad-4bd2-912b-7c350eecf1db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:51.610736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383f5941-32ad-4bd2-912b-7c350eecf1db None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:51.646283 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a044fe6-3355-46e5-b433-45891de5aec6 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:51.679905 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983ae11-a569-49af-a5a1-5ebf087487d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:51.680783 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983ae11-a569-49af-a5a1-5ebf087487d8 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:51.681068 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983ae11-a569-49af-a5a1-5ebf087487d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:51.681341 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983ae11-a569-49af-a5a1-5ebf087487d8 None admin] PATH_INFO: `/v3/users/d1134eaf564846f5990bd2b2934b7db7` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:51.682542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f983ae11-a569-49af-a5a1-5ebf087487d8 None admin] RBAC: Authorizing `identity:get_user(user_id=d1134eaf564846f5990bd2b2934b7db7)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:51.684173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f983ae11-a569-49af-a5a1-5ebf087487d8 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:51.685789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 8/16] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:52:51 2022] GET /identity/v3/users/d1134eaf564846f5990bd2b2934b7db7 => generated 247 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:52:51.691445 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983ae11-a569-49af-a5a1-5ebf087487d8 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:51.706421 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:52:51.709684 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 9/17] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:52:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 16:52:51.715093 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f5941-32ad-4bd2-912b-7c350eecf1db None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:51.718280 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:52:51.795521 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983ae11-a569-49af-a5a1-5ebf087487d8 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:51.815889 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383f5941-32ad-4bd2-912b-7c350eecf1db None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:51.829579 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e23bfb-8aa7-4dd3-bce8-e473900b8bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:51.830378 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e23bfb-8aa7-4dd3-bce8-e473900b8bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:51.830658 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e23bfb-8aa7-4dd3-bce8-e473900b8bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:51.830936 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e23bfb-8aa7-4dd3-bce8-e473900b8bb5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:51.832142 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e23bfb-8aa7-4dd3-bce8-e473900b8bb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:51.833617 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e23bfb-8aa7-4dd3-bce8-e473900b8bb5 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:51.835449 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 9/18] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:52:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:52:51.841117 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e23bfb-8aa7-4dd3-bce8-e473900b8bb5 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:51.851517 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9671aa6-8311-4573-876b-c571d8cf418e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:51.852326 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9671aa6-8311-4573-876b-c571d8cf418e None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:51.852608 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9671aa6-8311-4573-876b-c571d8cf418e None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:51.852953 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9671aa6-8311-4573-876b-c571d8cf418e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:51.871720 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9671aa6-8311-4573-876b-c571d8cf418e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:51.873142 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9671aa6-8311-4573-876b-c571d8cf418e None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:51.894848 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-e9671aa6-8311-4573-876b-c571d8cf418e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:52:51.894848 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:52:51.894848 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:52:51.894848 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:52:51.894848 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:52:51.894848 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:52:51.894848 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:52:51.894848 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:52:51.894848 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:52:51.894848 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:52:51.894848 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:52:51.894848 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:52:51.894848 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:52:51.894848 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:52:51.894848 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:52:51.895685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:52:51.895685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:52:51.895685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:52:51.895685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:52:51.895685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:52:51.895685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:52:51.895685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:52:51.895685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:52:51.895685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:52:51.895685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:52:51.895685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:52:51.895685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:52:51.895685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:52:51.895685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:52:51.895685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:52:51.895685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:52:51.895685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:52:51.899490 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:52:51.899490 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:52:51.899490 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:52:51.899490 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:52:51.899490 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 10/19] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:52:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:52:51.902054 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9671aa6-8311-4573-876b-c571d8cf418e None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:51.922783 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:52:51.942722 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e23bfb-8aa7-4dd3-bce8-e473900b8bb5 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:51.960825 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:52:51.978121 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c8e804-a511-422e-82ca-03becf506174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:51.978771 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8e804-a511-422e-82ca-03becf506174 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:51.979082 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8e804-a511-422e-82ca-03becf506174 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:51.979354 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c8e804-a511-422e-82ca-03becf506174 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:51.980163 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c8e804-a511-422e-82ca-03becf506174 None admin] RBAC: Adding query filter params (role.id=17bbe7cc1c3641509de5e9dca8534102, scope.domain.id=default, user.id=d1134eaf564846f5990bd2b2934b7db7) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:52:51.980468 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c8e804-a511-422e-82ca-03becf506174 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:51.982308 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c8e804-a511-422e-82ca-03becf506174 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:51.989140 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 10/20] 10.222.0.13 () {56 vars in 1322 bytes} [Mon May 23 16:52:51 2022] GET /identity/v3/role_assignments?user.id=d1134eaf564846f5990bd2b2934b7db7&scope.domain.id=default&role.id=17bbe7cc1c3641509de5e9dca8534102 => generated 233 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:52:52.007306 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9671aa6-8311-4573-876b-c571d8cf418e None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:52.048471 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873c2cdc-9912-452d-b2cc-ce60de372a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:52.049154 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c2cdc-9912-452d-b2cc-ce60de372a87 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:52.049435 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c2cdc-9912-452d-b2cc-ce60de372a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:52.049705 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873c2cdc-9912-452d-b2cc-ce60de372a87 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:52.050459 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c2cdc-9912-452d-b2cc-ce60de372a87 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:52:52.050762 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c2cdc-9912-452d-b2cc-ce60de372a87 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:52.052243 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873c2cdc-9912-452d-b2cc-ce60de372a87 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:52.072413 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 11/21] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:52:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:52:54.435967 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b187261-93fd-4052-9a30-850ea62485fc None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:54.436998 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b187261-93fd-4052-9a30-850ea62485fc None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:54.437498 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b187261-93fd-4052-9a30-850ea62485fc None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:54.439106 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 11/22] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:52:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:52:54.444963 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba9cbd-e8a8-470a-b138-65fcd9b4603d None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:54.445855 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba9cbd-e8a8-470a-b138-65fcd9b4603d None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:54.446326 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba9cbd-e8a8-470a-b138-65fcd9b4603d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:54.473846 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-e8ba9cbd-e8a8-470a-b138-65fcd9b4603d None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:52:54.475422 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-e8ba9cbd-e8a8-470a-b138-65fcd9b4603d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:52:54.581451 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 12/23] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:52:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:52:54.615738 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b187261-93fd-4052-9a30-850ea62485fc None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:54.722434 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c146dc-3d16-404c-a26c-1fe047afa0ba None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:54.722625 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c146dc-3d16-404c-a26c-1fe047afa0ba None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:54.722935 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c146dc-3d16-404c-a26c-1fe047afa0ba None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:54.724421 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 13/24] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:52:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:52:54.736943 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84573b5e-748a-4606-9ba1-ab60ed17303e None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:54.737280 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84573b5e-748a-4606-9ba1-ab60ed17303e None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:54.737500 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84573b5e-748a-4606-9ba1-ab60ed17303e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:54.752638 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b187261-93fd-4052-9a30-850ea62485fc None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:54.764814 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-84573b5e-748a-4606-9ba1-ab60ed17303e None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:52:54.766063 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-84573b5e-748a-4606-9ba1-ab60ed17303e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:52:54.829688 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc544c3-c25b-4652-abdc-4e8553f9c15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:54.830511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc544c3-c25b-4652-abdc-4e8553f9c15f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:54.830795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc544c3-c25b-4652-abdc-4e8553f9c15f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:54.831099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc544c3-c25b-4652-abdc-4e8553f9c15f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:54.856286 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc544c3-c25b-4652-abdc-4e8553f9c15f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:52:54.868332 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-5fc544c3-c25b-4652-abdc-4e8553f9c15f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:52:54.868332 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:52:54.868332 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:52:54.868332 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:52:54.868332 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:52:54.868332 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:52:54.868332 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:52:54.868332 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:52:54.868332 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:52:54.868332 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:52:54.868332 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:52:54.868332 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:52:54.868332 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:52:54.868332 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:52:54.868332 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:52:54.868332 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:52:54.869233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:52:54.869233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:52:54.869233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:52:54.869233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:52:54.869233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:52:54.869233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:52:54.869233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:52:54.869233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:52:54.869233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:52:54.869233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:52:54.869233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:52:54.869233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:52:54.869233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:52:54.869233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:52:54.869233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:52:54.869233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:52:54.869991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:52:54.869991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:52:54.869991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:52:54.869991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:52:54.869991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:52:54.869991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:52:54.869991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:52:54.869991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:52:54.869991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:52:54.869991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:52:54.869991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:52:54.869991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:52:54.872432 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 12/25] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:52:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 257 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:52:54.878179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc544c3-c25b-4652-abdc-4e8553f9c15f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:54.883337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:52:54.892815 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:52:54.932773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 14/26] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:52:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:52:54.938375 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e6f04-ca42-4f7c-83fe-08b6435fce14 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:54.938880 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e6f04-ca42-4f7c-83fe-08b6435fce14 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:54.939372 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87e6f04-ca42-4f7c-83fe-08b6435fce14 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:54.942161 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 15/27] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:52:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:52:54.946375 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2effc-1337-4ea3-b32a-290ec70832e3 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:54.946891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2effc-1337-4ea3-b32a-290ec70832e3 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:54.947400 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd2effc-1337-4ea3-b32a-290ec70832e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:54.974473 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-fdd2effc-1337-4ea3-b32a-290ec70832e3 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:52:54.976022 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-fdd2effc-1337-4ea3-b32a-290ec70832e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:52:55.065301 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc544c3-c25b-4652-abdc-4e8553f9c15f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:55.082272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 16/28] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:52:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 16:52:55.083587 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2effc-1337-4ea3-b32a-290ec70832e3 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:55.126544 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d7e98c-7b20-4e30-821b-d1af43624e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:55.127584 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7e98c-7b20-4e30-821b-d1af43624e67 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:55.127715 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7e98c-7b20-4e30-821b-d1af43624e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:55.127940 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d7e98c-7b20-4e30-821b-d1af43624e67 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:55.128703 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7e98c-7b20-4e30-821b-d1af43624e67 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:52:55.129012 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7e98c-7b20-4e30-821b-d1af43624e67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:55.130588 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d7e98c-7b20-4e30-821b-d1af43624e67 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:55.143094 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 13/29] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:52:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:52:55.145066 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e286c14-e11d-4c4e-b6d4-3ed4e68f929b None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:55.145366 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e286c14-e11d-4c4e-b6d4-3ed4e68f929b None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:55.145646 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e286c14-e11d-4c4e-b6d4-3ed4e68f929b None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:55.147564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 14/30] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:52:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:52:55.152447 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e286c14-e11d-4c4e-b6d4-3ed4e68f929b None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:55.195806 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd2effc-1337-4ea3-b32a-290ec70832e3 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:55.204791 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:52:55.245411 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:52:55.246721 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb310b95-b9c7-475a-ba31-3fe37cfc333b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:55.248815 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb310b95-b9c7-475a-ba31-3fe37cfc333b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:55.249417 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb310b95-b9c7-475a-ba31-3fe37cfc333b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:55.249837 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb310b95-b9c7-475a-ba31-3fe37cfc333b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:55.263685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb310b95-b9c7-475a-ba31-3fe37cfc333b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:55.265924 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb310b95-b9c7-475a-ba31-3fe37cfc333b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:55.287841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-bb310b95-b9c7-475a-ba31-3fe37cfc333b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:52:55.287841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:52:55.287841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:52:55.287841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:52:55.287841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:52:55.287841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:52:55.287841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:52:55.287841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:52:55.287841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:52:55.287841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:52:55.287841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:52:55.287841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:52:55.287841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:52:55.287841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:52:55.287841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:52:55.288735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:52:55.288735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:52:55.288735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:52:55.288735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:52:55.288735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:52:55.288735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:52:55.288735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:52:55.288735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:52:55.288735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:52:55.288735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:52:55.288735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:52:55.288735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:52:55.288735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:52:55.288735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:52:55.288735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:52:55.288735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:52:55.288735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:52:55.289524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:52:55.289524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:52:55.289524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:52:55.289524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:52:55.289524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 17/31] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:52:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:52:55.290699 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb310b95-b9c7-475a-ba31-3fe37cfc333b None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:55.292221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e286c14-e11d-4c4e-b6d4-3ed4e68f929b None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:55.333586 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760acfa9-13fa-48fc-8d04-e76c671791b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:55.335061 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760acfa9-13fa-48fc-8d04-e76c671791b1 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:55.335488 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760acfa9-13fa-48fc-8d04-e76c671791b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:55.335905 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760acfa9-13fa-48fc-8d04-e76c671791b1 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:55.336818 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760acfa9-13fa-48fc-8d04-e76c671791b1 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:55.339429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760acfa9-13fa-48fc-8d04-e76c671791b1 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:55.413585 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb310b95-b9c7-475a-ba31-3fe37cfc333b None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:55.449325 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194380c4-0c70-461d-89a0-586a01c7127c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:55.450389 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194380c4-0c70-461d-89a0-586a01c7127c None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:55.450814 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194380c4-0c70-461d-89a0-586a01c7127c None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:55.451255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194380c4-0c70-461d-89a0-586a01c7127c None admin] PATH_INFO: `/v3/users/d1134eaf564846f5990bd2b2934b7db7` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:55.452948 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194380c4-0c70-461d-89a0-586a01c7127c None admin] RBAC: Authorizing `identity:get_user(user_id=d1134eaf564846f5990bd2b2934b7db7)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:55.454564 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194380c4-0c70-461d-89a0-586a01c7127c None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:55.456765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 18/32] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:52:55 2022] GET /identity/v3/users/d1134eaf564846f5990bd2b2934b7db7 => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:52:55.458038 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01530de4-dc49-495a-9077-5d43d22ed497 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:55.458496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01530de4-dc49-495a-9077-5d43d22ed497 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:55.458939 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01530de4-dc49-495a-9077-5d43d22ed497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:55.482777 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-01530de4-dc49-495a-9077-5d43d22ed497 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:52:55.484244 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-01530de4-dc49-495a-9077-5d43d22ed497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:52:55.503586 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 15/33] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 23 16:52:55 2022] POST /identity/v3/roles => generated 219 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:52:55.504734 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760acfa9-13fa-48fc-8d04-e76c671791b1 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:55.508046 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:52:55.600257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 19/34] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:52:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:52:55.601158 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01530de4-dc49-495a-9077-5d43d22ed497 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:55.734257 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760acfa9-13fa-48fc-8d04-e76c671791b1 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:55.756674 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01530de4-dc49-495a-9077-5d43d22ed497 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:55.767850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:52:55.778754 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da81c37-b39a-4421-ac89-0dfed2b2d676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:55.780299 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da81c37-b39a-4421-ac89-0dfed2b2d676 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:55.781067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da81c37-b39a-4421-ac89-0dfed2b2d676 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:55.781625 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da81c37-b39a-4421-ac89-0dfed2b2d676 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:55.782607 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da81c37-b39a-4421-ac89-0dfed2b2d676 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:52:55.783416 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da81c37-b39a-4421-ac89-0dfed2b2d676 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:55.785869 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da81c37-b39a-4421-ac89-0dfed2b2d676 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:55.798000 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067ed19-2d4f-4cd2-934d-3298fab76d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:55.798816 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067ed19-2d4f-4cd2-934d-3298fab76d93 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:55.799125 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067ed19-2d4f-4cd2-934d-3298fab76d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:55.799397 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3067ed19-2d4f-4cd2-934d-3298fab76d93 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:55.800758 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3067ed19-2d4f-4cd2-934d-3298fab76d93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:55.802459 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3067ed19-2d4f-4cd2-934d-3298fab76d93 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:55.808389 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 16/35] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:52:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:52:55.809462 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da81c37-b39a-4421-ac89-0dfed2b2d676 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:55.809711 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 20/36] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:52:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:52:55.810564 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067ed19-2d4f-4cd2-934d-3298fab76d93 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:55.843120 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:52:55.936621 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3067ed19-2d4f-4cd2-934d-3298fab76d93 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:55.942600 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da81c37-b39a-4421-ac89-0dfed2b2d676 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:55.977848 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e6f472-c3c4-437e-aa26-e9f8362def44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:55.978751 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6f472-c3c4-437e-aa26-e9f8362def44 None admin] REQUEST_METHOD: `PUT` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:55.979066 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6f472-c3c4-437e-aa26-e9f8362def44 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:55.979343 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e6f472-c3c4-437e-aa26-e9f8362def44 None admin] PATH_INFO: `/v3/domains/default/users/d1134eaf564846f5990bd2b2934b7db7/roles/17bbe7cc1c3641509de5e9dca8534102` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:55.981602 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6f472-c3c4-437e-aa26-e9f8362def44 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=d1134eaf564846f5990bd2b2934b7db7, role_id=17bbe7cc1c3641509de5e9dca8534102)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:55.983102 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e6f472-c3c4-437e-aa26-e9f8362def44 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:55.986239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502754b-a7a5-4cc5-8dcc-28c9e76ab826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:55.987210 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502754b-a7a5-4cc5-8dcc-28c9e76ab826 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:55.987691 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502754b-a7a5-4cc5-8dcc-28c9e76ab826 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:55.988161 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6502754b-a7a5-4cc5-8dcc-28c9e76ab826 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:55.989237 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6502754b-a7a5-4cc5-8dcc-28c9e76ab826 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:55.991232 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6502754b-a7a5-4cc5-8dcc-28c9e76ab826 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:56.011252 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 17/37] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 23 16:52:55 2022] POST /identity/v3/roles => generated 225 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:52:56.012215 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502754b-a7a5-4cc5-8dcc-28c9e76ab826 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:56.065233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:52:56.112712 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 21/38] 10.222.0.13 () {58 vars in 1361 bytes} [Mon May 23 16:52:55 2022] PUT /identity/v3/domains/default/users/d1134eaf564846f5990bd2b2934b7db7/roles/17bbe7cc1c3641509de5e9dca8534102 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:52:56.132942 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6502754b-a7a5-4cc5-8dcc-28c9e76ab826 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:56.147982 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fabd94-438a-401d-af80-e23030a28a74 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:56.148657 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fabd94-438a-401d-af80-e23030a28a74 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:56.149145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fabd94-438a-401d-af80-e23030a28a74 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:56.153902 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 22/39] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:52:56 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:52:56.160409 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee9b53-20fe-4949-91a2-b43e187b03f5 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:56.160719 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee9b53-20fe-4949-91a2-b43e187b03f5 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:56.161094 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ee9b53-20fe-4949-91a2-b43e187b03f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:56.184386 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-41ee9b53-20fe-4949-91a2-b43e187b03f5 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:52:56.185997 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-41ee9b53-20fe-4949-91a2-b43e187b03f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:52:56.189874 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172fda3-2df4-4e22-b88d-c987b846f434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:56.190573 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172fda3-2df4-4e22-b88d-c987b846f434 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:56.190856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172fda3-2df4-4e22-b88d-c987b846f434 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:56.191158 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b172fda3-2df4-4e22-b88d-c987b846f434 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:56.191902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b172fda3-2df4-4e22-b88d-c987b846f434 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:56.194276 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b172fda3-2df4-4e22-b88d-c987b846f434 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:56.258389 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 18/40] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 23 16:52:56 2022] POST /identity/v3/projects => generated 301 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:52:56.319942 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 23/41] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:52:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:52:56.343425 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172fda3-2df4-4e22-b88d-c987b846f434 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:56.489573 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b172fda3-2df4-4e22-b88d-c987b846f434 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:56.499769 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:52:56.531093 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6525aa-5cc8-4368-be9f-942d4b065457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:56.531846 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6525aa-5cc8-4368-be9f-942d4b065457 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:56.532059 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6525aa-5cc8-4368-be9f-942d4b065457 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:56.532337 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6525aa-5cc8-4368-be9f-942d4b065457 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:56.533112 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6525aa-5cc8-4368-be9f-942d4b065457 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:56.534535 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6525aa-5cc8-4368-be9f-942d4b065457 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:56.551802 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 19/42] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 23 16:52:56 2022] POST /identity/v3/roles => generated 223 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:52:56.569929 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b651a-9db9-43a3-8b2a-95752cfe1389 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:56.570629 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b651a-9db9-43a3-8b2a-95752cfe1389 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:56.570629 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-486b651a-9db9-43a3-8b2a-95752cfe1389 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:56.572183 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 24/43] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:52:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:52:56.581301 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c09b5-55fc-475b-9a39-470785ce9aa8 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:56.581973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c09b5-55fc-475b-9a39-470785ce9aa8 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:56.582449 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35c09b5-55fc-475b-9a39-470785ce9aa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:56.601367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-e35c09b5-55fc-475b-9a39-470785ce9aa8 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:52:56.602966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e35c09b5-55fc-475b-9a39-470785ce9aa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:52:56.680142 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 20/44] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:52:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:52:56.703164 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b651a-9db9-43a3-8b2a-95752cfe1389 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:56.721707 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:52:56.817756 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-486b651a-9db9-43a3-8b2a-95752cfe1389 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:56.851547 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec66a63-3653-471e-b1d1-35c4255882f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:56.852350 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec66a63-3653-471e-b1d1-35c4255882f5 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:56.852632 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec66a63-3653-471e-b1d1-35c4255882f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:56.852900 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec66a63-3653-471e-b1d1-35c4255882f5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:56.854108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec66a63-3653-471e-b1d1-35c4255882f5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:56.855755 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec66a63-3653-471e-b1d1-35c4255882f5 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:56.857347 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 25/45] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:52:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:52:56.863270 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c09b5-55fc-475b-9a39-470785ce9aa8 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:56.951653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:52:56.965261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35c09b5-55fc-475b-9a39-470785ce9aa8 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:56.999754 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79cd558a-b34a-4b4a-9590-d5fba970da5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:57.000411 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd558a-b34a-4b4a-9590-d5fba970da5d None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:57.000694 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd558a-b34a-4b4a-9590-d5fba970da5d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:57.000973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79cd558a-b34a-4b4a-9590-d5fba970da5d None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:57.001703 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd558a-b34a-4b4a-9590-d5fba970da5d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:57.003187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79cd558a-b34a-4b4a-9590-d5fba970da5d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:57.029288 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 21/46] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 23 16:52:56 2022] POST /identity/v3/projects => generated 312 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 16:52:58.423265 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb83fbc-e5b5-4079-8991-d93a0f0d0519 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:58.423801 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb83fbc-e5b5-4079-8991-d93a0f0d0519 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:58.423801 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb83fbc-e5b5-4079-8991-d93a0f0d0519 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:58.425184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 26/47] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:52:58.431246 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8651bc-45b3-4d97-b255-4642b62149a2 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:58.431635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8651bc-45b3-4d97-b255-4642b62149a2 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:58.431820 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df8651bc-45b3-4d97-b255-4642b62149a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:58.449776 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-df8651bc-45b3-4d97-b255-4642b62149a2 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:52:58.450994 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-df8651bc-45b3-4d97-b255-4642b62149a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:52:58.520857 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 22/48] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:52:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:52:58.543910 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb83fbc-e5b5-4079-8991-d93a0f0d0519 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:58.633473 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:52:58.654306 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb83fbc-e5b5-4079-8991-d93a0f0d0519 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:58.688885 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a15a41-8c57-4809-a72c-32a54873cbb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:58.689601 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a15a41-8c57-4809-a72c-32a54873cbb4 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:58.689880 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a15a41-8c57-4809-a72c-32a54873cbb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:58.690151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a15a41-8c57-4809-a72c-32a54873cbb4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:58.701643 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a15a41-8c57-4809-a72c-32a54873cbb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:52:58.710346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-81a15a41-8c57-4809-a72c-32a54873cbb4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:52:58.710346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:52:58.710346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:52:58.710346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:52:58.710346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:52:58.710346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:52:58.710346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:52:58.710346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:52:58.710346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:52:58.710346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:52:58.710346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:52:58.710346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:52:58.710346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:52:58.710346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:52:58.710346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:52:58.710346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:52:58.711247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:52:58.711247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:52:58.711247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:52:58.711247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:52:58.711247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:52:58.711247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:52:58.711247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:52:58.711247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:52:58.711247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:52:58.711247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:52:58.711247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:52:58.711247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:52:58.711247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:52:58.711247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:52:58.711247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:52:58.711247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:52:58.712033 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:52:58.712033 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:52:58.712033 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:52:58.712033 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:52:58.712033 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:52:58.712033 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:52:58.712033 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:52:58.712033 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:52:58.712033 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:52:58.712033 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:52:58.712033 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:52:58.712033 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:52:58.712033 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 27/49] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:52:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:52:58.718853 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8651bc-45b3-4d97-b255-4642b62149a2 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:58.788695 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:52:58.821041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df8651bc-45b3-4d97-b255-4642b62149a2 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:58.855655 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d33a9-99fb-4bb6-8ca6-1cbd2f23b75b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:58.856351 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d33a9-99fb-4bb6-8ca6-1cbd2f23b75b None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:58.856631 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d33a9-99fb-4bb6-8ca6-1cbd2f23b75b None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:58.856903 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-944d33a9-99fb-4bb6-8ca6-1cbd2f23b75b None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:58.857645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944d33a9-99fb-4bb6-8ca6-1cbd2f23b75b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:52:58.857952 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944d33a9-99fb-4bb6-8ca6-1cbd2f23b75b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:58.859382 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-944d33a9-99fb-4bb6-8ca6-1cbd2f23b75b None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:58.867894 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 23/50] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:52:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:52:58.873508 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a15a41-8c57-4809-a72c-32a54873cbb4 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:58.977416 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a15a41-8c57-4809-a72c-32a54873cbb4 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:59.012477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d485e5-80b4-46d2-8464-5555f84ee6a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:59.013357 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d485e5-80b4-46d2-8464-5555f84ee6a7 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:59.013645 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d485e5-80b4-46d2-8464-5555f84ee6a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:59.013922 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d485e5-80b4-46d2-8464-5555f84ee6a7 None admin] PATH_INFO: `/v3/users/d1134eaf564846f5990bd2b2934b7db7` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:59.015197 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d485e5-80b4-46d2-8464-5555f84ee6a7 None admin] RBAC: Authorizing `identity:get_user(user_id=d1134eaf564846f5990bd2b2934b7db7)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:59.016668 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d485e5-80b4-46d2-8464-5555f84ee6a7 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:59.018334 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 28/51] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:52:58 2022] GET /identity/v3/users/d1134eaf564846f5990bd2b2934b7db7 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:52:59.024387 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d33a9-99fb-4bb6-8ca6-1cbd2f23b75b None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:59.130104 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-944d33a9-99fb-4bb6-8ca6-1cbd2f23b75b None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:59.160835 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:52:59.167300 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9400fd6b-708a-42ff-9e0f-49d911bb6a49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:59.168118 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400fd6b-708a-42ff-9e0f-49d911bb6a49 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:59.168405 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400fd6b-708a-42ff-9e0f-49d911bb6a49 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:59.168680 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9400fd6b-708a-42ff-9e0f-49d911bb6a49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:59.170015 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400fd6b-708a-42ff-9e0f-49d911bb6a49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:59.171566 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9400fd6b-708a-42ff-9e0f-49d911bb6a49 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:59.173175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 24/52] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:52:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:52:59.179122 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d485e5-80b4-46d2-8464-5555f84ee6a7 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:59.196704 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:52:59.224628 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15df05-db47-4df9-90b0-3d8a6cd45974 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:59.225010 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15df05-db47-4df9-90b0-3d8a6cd45974 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:59.225331 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b15df05-db47-4df9-90b0-3d8a6cd45974 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:59.227078 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 25/53] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:52:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:52:59.233063 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912e3b8-b036-4ac4-a1af-01629b07c195 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:59.233292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912e3b8-b036-4ac4-a1af-01629b07c195 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:59.233664 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a912e3b8-b036-4ac4-a1af-01629b07c195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:59.251034 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-a912e3b8-b036-4ac4-a1af-01629b07c195 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:52:59.252254 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a912e3b8-b036-4ac4-a1af-01629b07c195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:52:59.297678 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d485e5-80b4-46d2-8464-5555f84ee6a7 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:59.319835 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 26/54] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:52:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:52:59.331229 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2f213-3c57-4389-a0a2-67fd8e0c613d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:59.331891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2f213-3c57-4389-a0a2-67fd8e0c613d None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:59.332177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2f213-3c57-4389-a0a2-67fd8e0c613d None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:59.332448 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b2f213-3c57-4389-a0a2-67fd8e0c613d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:59.333278 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2f213-3c57-4389-a0a2-67fd8e0c613d None admin] RBAC: Adding query filter params (role.id=17bbe7cc1c3641509de5e9dca8534102, scope.domain.id=default, user.id=d1134eaf564846f5990bd2b2934b7db7) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:52:59.333772 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2f213-3c57-4389-a0a2-67fd8e0c613d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:59.335660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b2f213-3c57-4389-a0a2-67fd8e0c613d None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:59.342462 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912e3b8-b036-4ac4-a1af-01629b07c195 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:59.343691 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 29/55] 10.222.0.13 () {56 vars in 1322 bytes} [Mon May 23 16:52:59 2022] GET /identity/v3/role_assignments?user.id=d1134eaf564846f5990bd2b2934b7db7&scope.domain.id=default&role.id=17bbe7cc1c3641509de5e9dca8534102 => generated 530 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:52:59.449781 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a912e3b8-b036-4ac4-a1af-01629b07c195 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:59.484505 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:52:59.492189 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c391c84f-f6f6-4c67-80d4-7538c24fc269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:59.493018 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391c84f-f6f6-4c67-80d4-7538c24fc269 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:59.493303 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391c84f-f6f6-4c67-80d4-7538c24fc269 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:59.493580 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c391c84f-f6f6-4c67-80d4-7538c24fc269 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:59.494901 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391c84f-f6f6-4c67-80d4-7538c24fc269 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:59.496446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c391c84f-f6f6-4c67-80d4-7538c24fc269 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:59.498130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 27/56] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:52:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:52:59.504083 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2f213-3c57-4389-a0a2-67fd8e0c613d None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:52:59.586719 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:52:59.608722 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b2f213-3c57-4389-a0a2-67fd8e0c613d None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:52:59.647476 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f71cfb8-b635-4dda-9f08-d51603e36aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:52:59.648135 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71cfb8-b635-4dda-9f08-d51603e36aaf None admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:52:59.648416 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71cfb8-b635-4dda-9f08-d51603e36aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:52:59.648690 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f71cfb8-b635-4dda-9f08-d51603e36aaf None admin] PATH_INFO: `/v3/projects` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:52:59.649396 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71cfb8-b635-4dda-9f08-d51603e36aaf None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:52:59.650852 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f71cfb8-b635-4dda-9f08-d51603e36aaf None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:52:59.680987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 30/57] 10.222.0.13 () {60 vars in 1140 bytes} [Mon May 23 16:52:59 2022] POST /identity/v3/projects => generated 298 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:53:01.872986 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700357b-f444-4d62-a83f-4eb2a4c8b271 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:01.873539 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700357b-f444-4d62-a83f-4eb2a4c8b271 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:01.873539 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a700357b-f444-4d62-a83f-4eb2a4c8b271 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:01.874932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 28/58] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:01.881540 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63602b-2add-42cf-8ab8-d443333abe69 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:01.882210 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63602b-2add-42cf-8ab8-d443333abe69 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:01.882410 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa63602b-2add-42cf-8ab8-d443333abe69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:01.908640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-aa63602b-2add-42cf-8ab8-d443333abe69 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:01.910426 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-aa63602b-2add-42cf-8ab8-d443333abe69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:01.990330 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 31/59] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:02.015401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700357b-f444-4d62-a83f-4eb2a4c8b271 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:02.133528 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a700357b-f444-4d62-a83f-4eb2a4c8b271 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:02.169247 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5815cb-5ed0-40b0-bb00-620b40fdd7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:02.170035 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5815cb-5ed0-40b0-bb00-620b40fdd7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:02.170324 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5815cb-5ed0-40b0-bb00-620b40fdd7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:02.170607 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5815cb-5ed0-40b0-bb00-620b40fdd7b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:02.171862 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5815cb-5ed0-40b0-bb00-620b40fdd7b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:02.173317 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5815cb-5ed0-40b0-bb00-620b40fdd7b7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:02.174926 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 29/60] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:53:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:02.180728 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63602b-2add-42cf-8ab8-d443333abe69 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:02.241618 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:02.288734 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa63602b-2add-42cf-8ab8-d443333abe69 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:02.324977 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff38bba5-3e45-4c4e-87ae-199714ace07e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:02.325651 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38bba5-3e45-4c4e-87ae-199714ace07e None admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:02.325932 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38bba5-3e45-4c4e-87ae-199714ace07e None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:02.326210 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff38bba5-3e45-4c4e-87ae-199714ace07e None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:02.326959 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38bba5-3e45-4c4e-87ae-199714ace07e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:02.328429 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff38bba5-3e45-4c4e-87ae-199714ace07e None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:02.356121 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 32/61] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 23 16:53:02 2022] POST /identity/v3/users => generated 275 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:53:05.733829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a72e59-121d-401c-9a36-254c0b42445c None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:05.734653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a72e59-121d-401c-9a36-254c0b42445c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:05.735112 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a72e59-121d-401c-9a36-254c0b42445c None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:05.736639 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 30/62] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:05.742238 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c77d7-bac3-48a0-84fa-56e0ffb8bc21 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:05.743153 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c77d7-bac3-48a0-84fa-56e0ffb8bc21 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:05.743625 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458c77d7-bac3-48a0-84fa-56e0ffb8bc21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:05.771942 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-458c77d7-bac3-48a0-84fa-56e0ffb8bc21 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:05.773620 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-458c77d7-bac3-48a0-84fa-56e0ffb8bc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:05.838740 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3d5f3-a473-4d89-8e43-02b661f4ce2c None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:05.839433 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3d5f3-a473-4d89-8e43-02b661f4ce2c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:05.839925 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d3d5f3-a473-4d89-8e43-02b661f4ce2c None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:05.841578 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 31/63] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:53:05.847577 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5d392-4c08-4519-beb4-4451038d3718 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:05.848859 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5d392-4c08-4519-beb4-4451038d3718 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:05.849362 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f5d392-4c08-4519-beb4-4451038d3718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:05.883214 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-86f5d392-4c08-4519-beb4-4451038d3718 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:05.884452 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-86f5d392-4c08-4519-beb4-4451038d3718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:05.888494 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 33/64] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:05.911243 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c77d7-bac3-48a0-84fa-56e0ffb8bc21 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:05.980846 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 32/65] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:05.987369 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:06.007831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5d392-4c08-4519-beb4-4451038d3718 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:06.020805 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:06.101862 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458c77d7-bac3-48a0-84fa-56e0ffb8bc21 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:06.123363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f5d392-4c08-4519-beb4-4451038d3718 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:06.138591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96d432-636a-4a60-a820-a6dafeeea02a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:06.139455 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96d432-636a-4a60-a820-a6dafeeea02a None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:06.139739 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96d432-636a-4a60-a820-a6dafeeea02a None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:06.140100 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96d432-636a-4a60-a820-a6dafeeea02a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:06.151418 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96d432-636a-4a60-a820-a6dafeeea02a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:06.159355 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfe964a-1ce4-42a3-ac30-55f6043fb483 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:06.159477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-bf96d432-636a-4a60-a820-a6dafeeea02a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:53:06.159477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:06.159477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:06.159477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:06.159477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:06.159477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:06.159477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:06.159477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:06.159477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:06.159477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:06.159477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:06.159477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:06.159477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:06.159477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:06.159477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:06.159477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:06.160248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:06.160248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:06.160248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:06.160248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:06.160248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:06.160248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:06.160248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:06.160248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:06.160248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:06.160248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:06.160248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:06.160248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:06.160248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:06.160248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:06.160248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:06.160248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:06.161423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:06.161423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:06.161423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:06.161423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:06.161423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:06.161423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:06.161423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:06.161423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:06.161423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:06.161423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:06.161423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:53:06.161423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:06.162191 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe964a-1ce4-42a3-ac30-55f6043fb483 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:06.162191 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe964a-1ce4-42a3-ac30-55f6043fb483 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:06.162304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 34/66] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:53:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:53:06.162364 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfe964a-1ce4-42a3-ac30-55f6043fb483 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:06.162433 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdb5aa-0995-492e-818e-97a90fbd0127 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:06.162433 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdb5aa-0995-492e-818e-97a90fbd0127 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:06.162668 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18fdb5aa-0995-492e-818e-97a90fbd0127 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:06.163971 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 35/67] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:06.165032 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd780a-8cbc-4c87-8c17-34bb104eaa93 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:06.165308 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd780a-8cbc-4c87-8c17-34bb104eaa93 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:06.165734 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd780a-8cbc-4c87-8c17-34bb104eaa93 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:06.170669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 36/68] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:06.171767 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd780a-8cbc-4c87-8c17-34bb104eaa93 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:06.175818 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abfe964a-1ce4-42a3-ac30-55f6043fb483 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:06.183874 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-abfe964a-1ce4-42a3-ac30-55f6043fb483 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:06.183874 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:06.183874 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:06.183874 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:06.183874 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:06.183874 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:06.183874 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:06.183874 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:06.183874 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:06.183874 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:06.183874 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:06.183874 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:06.183874 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:06.183874 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:06.183874 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:06.183874 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:06.184704 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:06.184704 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:06.184704 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:06.184704 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:06.184704 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:06.184704 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:06.184704 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:06.184704 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:06.184704 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:06.184704 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:06.184704 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:06.184704 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:06.184704 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:06.184704 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:06.184704 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:06.184704 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:06.185479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:06.185479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:06.185479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:06.185479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:06.185479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:06.185479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:06.185479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:06.185479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:06.185479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:06.185479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:06.185479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:06.185479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:06.185479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 33/69] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:53:06.186562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7a208-589d-4886-9433-f23938b4d7f9 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:06.186562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7a208-589d-4886-9433-f23938b4d7f9 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:06.186783 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a7a208-589d-4886-9433-f23938b4d7f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:06.204323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-06a7a208-589d-4886-9433-f23938b4d7f9 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:06.205536 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-06a7a208-589d-4886-9433-f23938b4d7f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:06.268817 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:06.285014 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd780a-8cbc-4c87-8c17-34bb104eaa93 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:06.286716 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:06.287529 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 34/70] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:06.292581 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902d63e-3acd-48ff-88d2-a535205b9e13 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:06.292951 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902d63e-3acd-48ff-88d2-a535205b9e13 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:06.293299 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2902d63e-3acd-48ff-88d2-a535205b9e13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:06.314594 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-2902d63e-3acd-48ff-88d2-a535205b9e13 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:06.315789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-2902d63e-3acd-48ff-88d2-a535205b9e13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:06.325395 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd793d-0c17-4b18-8763-afac4d511b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:06.326399 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd793d-0c17-4b18-8763-afac4d511b6e None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:06.326854 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd793d-0c17-4b18-8763-afac4d511b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:06.327415 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dd793d-0c17-4b18-8763-afac4d511b6e None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:06.328354 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd793d-0c17-4b18-8763-afac4d511b6e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:06.328833 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd793d-0c17-4b18-8763-afac4d511b6e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:06.330633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dd793d-0c17-4b18-8763-afac4d511b6e None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:06.343843 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 37/71] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:06.344916 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd793d-0c17-4b18-8763-afac4d511b6e None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:06.394382 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 35/72] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:06.395567 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902d63e-3acd-48ff-88d2-a535205b9e13 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:06.455415 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dd793d-0c17-4b18-8763-afac4d511b6e None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:06.493014 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973767bd-2b1f-478a-9c0d-fcfa896732ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:06.493723 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973767bd-2b1f-478a-9c0d-fcfa896732ae None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:06.494009 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973767bd-2b1f-478a-9c0d-fcfa896732ae None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:06.494288 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973767bd-2b1f-478a-9c0d-fcfa896732ae None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:06.495112 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973767bd-2b1f-478a-9c0d-fcfa896732ae None admin] RBAC: Adding query filter params (name=member) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:06.495411 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973767bd-2b1f-478a-9c0d-fcfa896732ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:06.496892 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-973767bd-2b1f-478a-9c0d-fcfa896732ae None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:06.503773 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2902d63e-3acd-48ff-88d2-a535205b9e13 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:06.505332 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 38/73] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:06.506134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973767bd-2b1f-478a-9c0d-fcfa896732ae None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:06.525086 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:06.536885 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:06.544222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d3237-010e-452c-aaa2-cbdde5966f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:06.545035 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d3237-010e-452c-aaa2-cbdde5966f25 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:06.545323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d3237-010e-452c-aaa2-cbdde5966f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:06.545599 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d3237-010e-452c-aaa2-cbdde5966f25 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:06.556618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d3237-010e-452c-aaa2-cbdde5966f25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:06.564230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-496d3237-010e-452c-aaa2-cbdde5966f25 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:06.564230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:06.564230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:06.564230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:06.564230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:06.564230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:06.564230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:06.564230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:06.564230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:06.564230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:06.564230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:06.564230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:06.564230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:06.564230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:06.564230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:06.564230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:06.565159 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:06.565159 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:06.565159 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:06.565159 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:06.565159 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:06.565159 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:06.565159 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:06.565159 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:06.565159 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:06.565159 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:06.565159 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:06.565159 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:06.565159 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:06.565159 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:06.565159 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:06.565159 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:06.565932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:06.565932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:06.565932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:06.565932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:06.565932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:06.565932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:06.565932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:06.565932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:06.565932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:06.565932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:06.565932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:06.565932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:06.565932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 36/74] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:06.566689 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74186b1d-009a-4355-98e3-97d8aa39a699 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:06.566995 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74186b1d-009a-4355-98e3-97d8aa39a699 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:06.567296 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74186b1d-009a-4355-98e3-97d8aa39a699 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:06.570963 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 37/75] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:06.570963 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74186b1d-009a-4355-98e3-97d8aa39a699 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:06.611759 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973767bd-2b1f-478a-9c0d-fcfa896732ae None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:06.644917 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f1265-de93-4cec-9a85-6901a9d5d062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:06.645798 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f1265-de93-4cec-9a85-6901a9d5d062 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:06.646083 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f1265-de93-4cec-9a85-6901a9d5d062 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:06.646358 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5f1265-de93-4cec-9a85-6901a9d5d062 None admin] PATH_INFO: `/v3/users/d1134eaf564846f5990bd2b2934b7db7` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:06.647634 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f1265-de93-4cec-9a85-6901a9d5d062 None admin] RBAC: Authorizing `identity:get_user(user_id=d1134eaf564846f5990bd2b2934b7db7)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:06.649084 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5f1265-de93-4cec-9a85-6901a9d5d062 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:06.650923 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 39/76] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:06 2022] GET /identity/v3/users/d1134eaf564846f5990bd2b2934b7db7 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:06.651544 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f1265-de93-4cec-9a85-6901a9d5d062 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:06.673944 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74186b1d-009a-4355-98e3-97d8aa39a699 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:06.708652 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee7b4b-b5cc-43dd-9c0f-36c7d3761edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:06.709474 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee7b4b-b5cc-43dd-9c0f-36c7d3761edd None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:06.709763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee7b4b-b5cc-43dd-9c0f-36c7d3761edd None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:06.710042 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee7b4b-b5cc-43dd-9c0f-36c7d3761edd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:06.720612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ee7b4b-b5cc-43dd-9c0f-36c7d3761edd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:06.729208 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-33ee7b4b-b5cc-43dd-9c0f-36c7d3761edd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:53:06.729208 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:06.729208 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:06.729208 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:06.729208 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:06.729208 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:06.729208 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:06.729208 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:06.729208 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:06.729208 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:06.729208 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:06.729208 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:06.729208 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:06.729208 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:06.729208 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:06.729208 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:06.730146 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:06.730146 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:06.730146 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:06.730146 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:06.730146 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:06.730146 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:06.730146 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:06.730146 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:06.730146 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:06.730146 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:06.730146 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:06.730146 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:06.730146 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:06.730146 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:06.730146 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:06.730146 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:06.730901 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:06.730901 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:06.730901 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:06.730901 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:06.730901 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:06.730901 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:06.730901 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:06.730901 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:06.730901 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:06.730901 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:06.730901 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:53:06.730901 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:06.730901 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 38/77] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 23 16:53:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:06.731562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee7b4b-b5cc-43dd-9c0f-36c7d3761edd None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:06.752502 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:06.760050 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5f1265-de93-4cec-9a85-6901a9d5d062 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:06.771244 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:06.794207 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc18294-f026-4495-aaf6-a6fd1f9a02f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:06.795047 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc18294-f026-4495-aaf6-a6fd1f9a02f3 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:06.795328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc18294-f026-4495-aaf6-a6fd1f9a02f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:06.795601 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc18294-f026-4495-aaf6-a6fd1f9a02f3 None admin] PATH_INFO: `/v3/users/99a4070a60cc4ae8a1981ac1c10668fe` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:06.811121 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc18294-f026-4495-aaf6-a6fd1f9a02f3 None admin] RBAC: Authorizing `identity:get_user(user_id=99a4070a60cc4ae8a1981ac1c10668fe)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:06.812595 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc18294-f026-4495-aaf6-a6fd1f9a02f3 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:06.814278 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 40/78] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:06 2022] GET /identity/v3/users/99a4070a60cc4ae8a1981ac1c10668fe => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:06.815565 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953cfd5-a61f-4492-9ff2-f195b008cb5c None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:06.815844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953cfd5-a61f-4492-9ff2-f195b008cb5c None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:06.816126 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b953cfd5-a61f-4492-9ff2-f195b008cb5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:06.832135 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-b953cfd5-a61f-4492-9ff2-f195b008cb5c None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:06.833328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-b953cfd5-a61f-4492-9ff2-f195b008cb5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:06.836557 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee7b4b-b5cc-43dd-9c0f-36c7d3761edd None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:06.869150 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad27a5-abb9-4728-8783-65d4268f3686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:06.869879 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad27a5-abb9-4728-8783-65d4268f3686 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:06.870107 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad27a5-abb9-4728-8783-65d4268f3686 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:06.870387 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad27a5-abb9-4728-8783-65d4268f3686 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:06.871181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad27a5-abb9-4728-8783-65d4268f3686 None admin] RBAC: Adding query filter params (name=member) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:06.871495 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad27a5-abb9-4728-8783-65d4268f3686 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:06.872913 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cad27a5-abb9-4728-8783-65d4268f3686 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:06.880994 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 39/79] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:06.881977 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad27a5-abb9-4728-8783-65d4268f3686 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:06.910291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 41/80] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:06.911165 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953cfd5-a61f-4492-9ff2-f195b008cb5c None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:06.978360 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:06.983487 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cad27a5-abb9-4728-8783-65d4268f3686 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:06.993738 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:07.013530 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b953cfd5-a61f-4492-9ff2-f195b008cb5c None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:07.018762 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be356e32-334d-4962-aff5-4b8ca802261f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:07.019460 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be356e32-334d-4962-aff5-4b8ca802261f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:07.019740 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be356e32-334d-4962-aff5-4b8ca802261f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:07.020008 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be356e32-334d-4962-aff5-4b8ca802261f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:07.050236 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7e07b-229f-4adc-a8a3-9e2d16ff4524 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:07.050847 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7e07b-229f-4adc-a8a3-9e2d16ff4524 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:07.051164 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7e07b-229f-4adc-a8a3-9e2d16ff4524 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:07.051444 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec7e07b-229f-4adc-a8a3-9e2d16ff4524 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:07.052197 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7e07b-229f-4adc-a8a3-9e2d16ff4524 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:07.052506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7e07b-229f-4adc-a8a3-9e2d16ff4524 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:07.054413 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec7e07b-229f-4adc-a8a3-9e2d16ff4524 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:07.062315 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 42/81] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:53:06 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:07.063144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7e07b-229f-4adc-a8a3-9e2d16ff4524 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:07.086016 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be356e32-334d-4962-aff5-4b8ca802261f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:07.087548 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be356e32-334d-4962-aff5-4b8ca802261f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:07.163808 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec7e07b-229f-4adc-a8a3-9e2d16ff4524 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:07.179426 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:07.184333 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 40/82] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:53:07.185269 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be356e32-334d-4962-aff5-4b8ca802261f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:07.203092 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1312de2-1476-49a9-94c0-366602dd81f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:07.203758 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1312de2-1476-49a9-94c0-366602dd81f4 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:07.204039 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1312de2-1476-49a9-94c0-366602dd81f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:07.204316 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1312de2-1476-49a9-94c0-366602dd81f4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:07.220302 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:07.274233 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1312de2-1476-49a9-94c0-366602dd81f4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:07.275743 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1312de2-1476-49a9-94c0-366602dd81f4 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:07.287601 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be356e32-334d-4962-aff5-4b8ca802261f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:07.321815 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57748a-f02b-4307-b8d6-b9bbd60f69b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:07.322611 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57748a-f02b-4307-b8d6-b9bbd60f69b8 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:07.322883 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57748a-f02b-4307-b8d6-b9bbd60f69b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:07.323178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af57748a-f02b-4307-b8d6-b9bbd60f69b8 None admin] PATH_INFO: `/v3/users/99a4070a60cc4ae8a1981ac1c10668fe` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:07.324378 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57748a-f02b-4307-b8d6-b9bbd60f69b8 None admin] RBAC: Authorizing `identity:get_user(user_id=99a4070a60cc4ae8a1981ac1c10668fe)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:07.325836 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af57748a-f02b-4307-b8d6-b9bbd60f69b8 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:07.327500 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 41/83] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:07 2022] GET /identity/v3/users/99a4070a60cc4ae8a1981ac1c10668fe => generated 275 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:07.328254 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57748a-f02b-4307-b8d6-b9bbd60f69b8 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:07.375042 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 43/84] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:53:07.375935 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1312de2-1476-49a9-94c0-366602dd81f4 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:07.391968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:07.423186 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:07.426562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af57748a-f02b-4307-b8d6-b9bbd60f69b8 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:07.459418 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f79138-ae7f-4247-abbc-daf7e8ab3387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:07.460260 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f79138-ae7f-4247-abbc-daf7e8ab3387 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:07.460526 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f79138-ae7f-4247-abbc-daf7e8ab3387 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:07.460814 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f79138-ae7f-4247-abbc-daf7e8ab3387 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:07.461997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f79138-ae7f-4247-abbc-daf7e8ab3387 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:07.463490 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f79138-ae7f-4247-abbc-daf7e8ab3387 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:07.465412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 42/85] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:53:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:07.466207 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f79138-ae7f-4247-abbc-daf7e8ab3387 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:07.480178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1312de2-1476-49a9-94c0-366602dd81f4 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:07.516399 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51358892-a2b6-4f4d-af97-8a4724074369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:07.517275 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51358892-a2b6-4f4d-af97-8a4724074369 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:07.517559 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51358892-a2b6-4f4d-af97-8a4724074369 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:07.517831 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51358892-a2b6-4f4d-af97-8a4724074369 None admin] PATH_INFO: `/v3/users/99a4070a60cc4ae8a1981ac1c10668fe` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:07.519068 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51358892-a2b6-4f4d-af97-8a4724074369 None admin] RBAC: Authorizing `identity:get_user(user_id=99a4070a60cc4ae8a1981ac1c10668fe)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:07.520527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51358892-a2b6-4f4d-af97-8a4724074369 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:07.522182 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 44/86] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:07 2022] GET /identity/v3/users/99a4070a60cc4ae8a1981ac1c10668fe => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:07.523261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51358892-a2b6-4f4d-af97-8a4724074369 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:07.569887 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f79138-ae7f-4247-abbc-daf7e8ab3387 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:07.605648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b93e6-4c56-4ad6-88f1-6f7db39d178c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:07.606527 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b93e6-4c56-4ad6-88f1-6f7db39d178c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:07.606814 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b93e6-4c56-4ad6-88f1-6f7db39d178c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:07.607121 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084b93e6-4c56-4ad6-88f1-6f7db39d178c None admin] PATH_INFO: `/v3/projects/f29295d0fb6141208e3a094c3be29944` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:07.608396 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084b93e6-4c56-4ad6-88f1-6f7db39d178c None admin] RBAC: Authorizing `identity:get_project(project_id=f29295d0fb6141208e3a094c3be29944)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:07.609869 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084b93e6-4c56-4ad6-88f1-6f7db39d178c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:07.611626 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 43/87] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:07 2022] GET /identity/v3/projects/f29295d0fb6141208e3a094c3be29944 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:07.612390 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b93e6-4c56-4ad6-88f1-6f7db39d178c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:07.615056 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:07.626295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51358892-a2b6-4f4d-af97-8a4724074369 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:07.639692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:07.662299 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db63b5d-14f5-481c-8b2e-59f29ec672ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:07.663088 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db63b5d-14f5-481c-8b2e-59f29ec672ea None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:07.663368 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db63b5d-14f5-481c-8b2e-59f29ec672ea None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:07.663650 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db63b5d-14f5-481c-8b2e-59f29ec672ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:07.712152 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084b93e6-4c56-4ad6-88f1-6f7db39d178c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:07.731377 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db63b5d-14f5-481c-8b2e-59f29ec672ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:07.732859 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db63b5d-14f5-481c-8b2e-59f29ec672ea None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:07.745122 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbbd33-b5cc-40bb-95c8-a2efd2527294 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:07.745985 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbbd33-b5cc-40bb-95c8-a2efd2527294 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:07.746292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbbd33-b5cc-40bb-95c8-a2efd2527294 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:07.746555 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbbbd33-b5cc-40bb-95c8-a2efd2527294 None admin] PATH_INFO: `/v3/projects/f29295d0fb6141208e3a094c3be29944` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:07.747752 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbbd33-b5cc-40bb-95c8-a2efd2527294 None admin] RBAC: Authorizing `identity:get_project(project_id=f29295d0fb6141208e3a094c3be29944)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:07.749187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbbbd33-b5cc-40bb-95c8-a2efd2527294 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:07.750874 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 44/88] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:07 2022] GET /identity/v3/projects/f29295d0fb6141208e3a094c3be29944 => generated 298 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:07.751626 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbbd33-b5cc-40bb-95c8-a2efd2527294 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:07.821594 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:07.834790 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 45/89] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:53:07.835692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db63b5d-14f5-481c-8b2e-59f29ec672ea None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:07.849970 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:07.852733 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbbbd33-b5cc-40bb-95c8-a2efd2527294 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:07.888350 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf195dc1-4736-408f-8c1a-72f87839a020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:07.888956 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf195dc1-4736-408f-8c1a-72f87839a020 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:07.889245 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf195dc1-4736-408f-8c1a-72f87839a020 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:07.889523 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf195dc1-4736-408f-8c1a-72f87839a020 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:07.890237 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf195dc1-4736-408f-8c1a-72f87839a020 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:07.891748 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf195dc1-4736-408f-8c1a-72f87839a020 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:07.918338 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 45/90] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 23 16:53:07 2022] POST /identity/v3/users => generated 289 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:53:07.918338 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf195dc1-4736-408f-8c1a-72f87839a020 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:07.948885 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db63b5d-14f5-481c-8b2e-59f29ec672ea None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:07.988472 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a42551-051d-4e15-be93-a3518e43b474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:07.989154 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a42551-051d-4e15-be93-a3518e43b474 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:07.989443 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a42551-051d-4e15-be93-a3518e43b474 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:07.989723 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a42551-051d-4e15-be93-a3518e43b474 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:08.034781 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf195dc1-4736-408f-8c1a-72f87839a020 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:08.063436 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:08.071092 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a42551-051d-4e15-be93-a3518e43b474 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:08.072770 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05a42551-051d-4e15-be93-a3518e43b474 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:08.074209 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359f00b-d053-4320-99b2-57e7531d1434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:08.074994 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359f00b-d053-4320-99b2-57e7531d1434 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:08.075219 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359f00b-d053-4320-99b2-57e7531d1434 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:08.075528 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3359f00b-d053-4320-99b2-57e7531d1434 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:08.077128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3359f00b-d053-4320-99b2-57e7531d1434 None admin] RBAC: Adding query filter params (role.id=17bbe7cc1c3641509de5e9dca8534102, scope.project.id=f29295d0fb6141208e3a094c3be29944, user.id=d1134eaf564846f5990bd2b2934b7db7) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:08.077128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3359f00b-d053-4320-99b2-57e7531d1434 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:08.078276 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3359f00b-d053-4320-99b2-57e7531d1434 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:08.084607 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 46/91] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:53:07 2022] GET /identity/v3/role_assignments?user.id=d1134eaf564846f5990bd2b2934b7db7&scope.project.id=f29295d0fb6141208e3a094c3be29944&role.id=17bbe7cc1c3641509de5e9dca8534102 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:08.085836 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359f00b-d053-4320-99b2-57e7531d1434 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:08.102022 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:08.229630 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 46/92] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:53:08.230839 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a42551-051d-4e15-be93-a3518e43b474 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:08.243819 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3359f00b-d053-4320-99b2-57e7531d1434 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:08.300795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a7b39-3df7-4307-80c9-39992c6945db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:08.301479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a7b39-3df7-4307-80c9-39992c6945db None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:08.301761 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a7b39-3df7-4307-80c9-39992c6945db None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:08.302042 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-864a7b39-3df7-4307-80c9-39992c6945db None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:08.302889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864a7b39-3df7-4307-80c9-39992c6945db None admin] RBAC: Adding query filter params (role.id=72cc26a1a66c4fc183b39fad4cb38431, scope.project.id=f29295d0fb6141208e3a094c3be29944, user.id=99a4070a60cc4ae8a1981ac1c10668fe) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:08.305147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864a7b39-3df7-4307-80c9-39992c6945db None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:08.306655 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-864a7b39-3df7-4307-80c9-39992c6945db None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:08.314751 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 47/93] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:53:08 2022] GET /identity/v3/role_assignments?user.id=99a4070a60cc4ae8a1981ac1c10668fe&scope.project.id=f29295d0fb6141208e3a094c3be29944&role.id=72cc26a1a66c4fc183b39fad4cb38431 => generated 259 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:08.315644 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a7b39-3df7-4307-80c9-39992c6945db None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:08.375470 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a42551-051d-4e15-be93-a3518e43b474 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:08.392974 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:08.406722 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:08.471230 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5d406-846b-4902-8251-d84f33b031e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:08.472132 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5d406-846b-4902-8251-d84f33b031e1 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:08.472412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5d406-846b-4902-8251-d84f33b031e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:08.472688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f5d406-846b-4902-8251-d84f33b031e1 None admin] PATH_INFO: `/v3/projects/15bd2384217841e5bc06a9047dbfb59d` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:08.475893 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5d406-846b-4902-8251-d84f33b031e1 None admin] RBAC: Authorizing `identity:get_project(project_id=15bd2384217841e5bc06a9047dbfb59d)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:08.483810 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f5d406-846b-4902-8251-d84f33b031e1 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:08.485881 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 47/94] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:08 2022] GET /identity/v3/projects/15bd2384217841e5bc06a9047dbfb59d => generated 312 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:08.492552 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5d406-846b-4902-8251-d84f33b031e1 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:08.529067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-864a7b39-3df7-4307-80c9-39992c6945db None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:08.599926 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917d165-dd61-41b7-b8e4-251c3269cb75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:08.601341 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917d165-dd61-41b7-b8e4-251c3269cb75 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:08.601612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917d165-dd61-41b7-b8e4-251c3269cb75 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:08.601890 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917d165-dd61-41b7-b8e4-251c3269cb75 None admin] PATH_INFO: `/v3/projects/f29295d0fb6141208e3a094c3be29944` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:08.603162 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2917d165-dd61-41b7-b8e4-251c3269cb75 None admin] RBAC: Authorizing `identity:get_project(project_id=f29295d0fb6141208e3a094c3be29944)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:08.604641 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2917d165-dd61-41b7-b8e4-251c3269cb75 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:08.606369 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 48/95] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:08 2022] GET /identity/v3/projects/f29295d0fb6141208e3a094c3be29944 => generated 298 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:08.612188 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917d165-dd61-41b7-b8e4-251c3269cb75 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:08.657303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f5d406-846b-4902-8251-d84f33b031e1 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:08.710824 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3cc47c-c35f-4777-8d98-cd7669a2add2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:08.711537 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cc47c-c35f-4777-8d98-cd7669a2add2 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:08.711947 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cc47c-c35f-4777-8d98-cd7669a2add2 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:08.712235 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3cc47c-c35f-4777-8d98-cd7669a2add2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:08.713091 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3cc47c-c35f-4777-8d98-cd7669a2add2 None admin] RBAC: Adding query filter params (role.id=72cc26a1a66c4fc183b39fad4cb38431, scope.project.id=15bd2384217841e5bc06a9047dbfb59d, user.id=99a4070a60cc4ae8a1981ac1c10668fe) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:08.713410 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3cc47c-c35f-4777-8d98-cd7669a2add2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:08.714864 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3cc47c-c35f-4777-8d98-cd7669a2add2 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:08.729018 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 48/96] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:53:08 2022] GET /identity/v3/role_assignments?user.id=99a4070a60cc4ae8a1981ac1c10668fe&scope.project.id=15bd2384217841e5bc06a9047dbfb59d&role.id=72cc26a1a66c4fc183b39fad4cb38431 => generated 259 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:08.817988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917d165-dd61-41b7-b8e4-251c3269cb75 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:08.849773 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:08.866961 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555b05e7-1c18-4fd8-a3de-9f4ab5d60342 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:08.867710 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b05e7-1c18-4fd8-a3de-9f4ab5d60342 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:08.867995 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b05e7-1c18-4fd8-a3de-9f4ab5d60342 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:08.868271 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555b05e7-1c18-4fd8-a3de-9f4ab5d60342 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:08.869122 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b05e7-1c18-4fd8-a3de-9f4ab5d60342 None admin] RBAC: Adding query filter params (role.id=6c30aca17fad4c4fa40b9461c5bda291, scope.project.id=f29295d0fb6141208e3a094c3be29944, user.id=99a4070a60cc4ae8a1981ac1c10668fe) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:08.869437 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b05e7-1c18-4fd8-a3de-9f4ab5d60342 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:08.871031 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555b05e7-1c18-4fd8-a3de-9f4ab5d60342 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:08.883846 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 49/97] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:53:08 2022] GET /identity/v3/role_assignments?user.id=99a4070a60cc4ae8a1981ac1c10668fe&scope.project.id=f29295d0fb6141208e3a094c3be29944&role.id=6c30aca17fad4c4fa40b9461c5bda291 => generated 259 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:10.860293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca1ba4-fbae-4900-8441-1c1ff07e6db1 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:10.861355 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca1ba4-fbae-4900-8441-1c1ff07e6db1 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:10.861841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ca1ba4-fbae-4900-8441-1c1ff07e6db1 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:10.863492 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 49/98] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:10.881047 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889378b-6650-4c85-9e8f-bb1c65b13b2a None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:10.882064 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889378b-6650-4c85-9e8f-bb1c65b13b2a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:10.882539 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6889378b-6650-4c85-9e8f-bb1c65b13b2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:10.905703 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-6889378b-6650-4c85-9e8f-bb1c65b13b2a None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:10.907218 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-6889378b-6650-4c85-9e8f-bb1c65b13b2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:11.013227 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 50/99] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:11.050527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca1ba4-fbae-4900-8441-1c1ff07e6db1 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:11.100625 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:11.165277 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ca1ba4-fbae-4900-8441-1c1ff07e6db1 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:11.214898 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa10de-4571-4d5c-a252-98a4010631b1 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:11.215621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa10de-4571-4d5c-a252-98a4010631b1 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:11.216139 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2aa10de-4571-4d5c-a252-98a4010631b1 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:11.218068 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 51/100] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:53:11.225640 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b6203-d559-4623-aaa6-6fcc4622bddd None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:11.226236 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b6203-d559-4623-aaa6-6fcc4622bddd None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:11.226709 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b6203-d559-4623-aaa6-6fcc4622bddd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:11.238899 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f992be96-376c-4071-a014-18a9e894132b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:11.240099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992be96-376c-4071-a014-18a9e894132b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:11.240581 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992be96-376c-4071-a014-18a9e894132b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:11.241478 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f992be96-376c-4071-a014-18a9e894132b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:11.244212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992be96-376c-4071-a014-18a9e894132b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:11.245915 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f992be96-376c-4071-a014-18a9e894132b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:11.248062 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 50/101] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:53:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:11.256604 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-131b6203-d559-4623-aaa6-6fcc4622bddd None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:11.258070 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-131b6203-d559-4623-aaa6-6fcc4622bddd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:11.262412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e573e41-ee08-4265-a460-e80a8984d7d2 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:11.262958 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e573e41-ee08-4265-a460-e80a8984d7d2 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:11.263575 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e573e41-ee08-4265-a460-e80a8984d7d2 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:11.265356 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 51/102] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:53:11.266282 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e573e41-ee08-4265-a460-e80a8984d7d2 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:11.385914 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 52/103] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:11.387132 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd67921-6312-4a49-bb88-4c852491904d None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:11.387528 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd67921-6312-4a49-bb88-4c852491904d None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:11.387808 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd67921-6312-4a49-bb88-4c852491904d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:11.405733 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e573e41-ee08-4265-a460-e80a8984d7d2 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:11.423812 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:11.424748 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-7fd67921-6312-4a49-bb88-4c852491904d None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:11.426244 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7fd67921-6312-4a49-bb88-4c852491904d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:11.446722 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5be800c-671c-4fc4-bad4-a514e3612f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:11.447656 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be800c-671c-4fc4-bad4-a514e3612f9e None admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:11.448302 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be800c-671c-4fc4-bad4-a514e3612f9e None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:11.448582 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5be800c-671c-4fc4-bad4-a514e3612f9e None admin] PATH_INFO: `/v3/projects` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:11.449294 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be800c-671c-4fc4-bad4-a514e3612f9e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:11.450774 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5be800c-671c-4fc4-bad4-a514e3612f9e None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:11.479083 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:11.488409 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 52/104] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 23 16:53:11 2022] POST /identity/v3/projects => generated 302 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:53:11.489733 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14989a79-87aa-45dd-a1e1-cbabfd21d6df None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:11.490018 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14989a79-87aa-45dd-a1e1-cbabfd21d6df None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:11.490291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14989a79-87aa-45dd-a1e1-cbabfd21d6df None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:11.492022 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 53/105] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:11.492757 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14989a79-87aa-45dd-a1e1-cbabfd21d6df None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:11.523966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 53/106] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:11.525585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e09f1-a696-475f-861f-9391ce837813 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:11.525989 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e09f1-a696-475f-861f-9391ce837813 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:11.526274 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212e09f1-a696-475f-861f-9391ce837813 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:11.557188 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-212e09f1-a696-475f-861f-9391ce837813 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:11.558504 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-212e09f1-a696-475f-861f-9391ce837813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:11.627179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14989a79-87aa-45dd-a1e1-cbabfd21d6df None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:11.673446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 54/107] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:11.674265 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9c35e-7250-4338-9cc6-4ff595b2f916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:11.674404 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e09f1-a696-475f-861f-9391ce837813 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:11.675456 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9c35e-7250-4338-9cc6-4ff595b2f916 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:11.675883 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9c35e-7250-4338-9cc6-4ff595b2f916 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:11.676303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f9c35e-7250-4338-9cc6-4ff595b2f916 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:11.696483 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41f9c35e-7250-4338-9cc6-4ff595b2f916 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:11.711714 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-41f9c35e-7250-4338-9cc6-4ff595b2f916 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:11.711714 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:11.711714 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:11.711714 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:11.711714 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:11.711714 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:11.711714 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:11.711714 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:11.711714 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:11.711714 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:11.711714 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:11.711714 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:11.711714 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:11.711714 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:11.711714 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:11.711714 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:11.712633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:11.712633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:11.712633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:11.712633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:11.712633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:11.712633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:11.712633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:11.712633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:11.712633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:11.712633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:11.712633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:11.712633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:11.712633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:11.712633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:11.712633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:11.712633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:11.714105 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:11.714105 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:11.714105 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:11.714105 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:11.714105 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:11.714105 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:11.714105 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:11.714105 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:11.714105 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:11.714105 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:11.714105 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:11.714105 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:11.714105 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 54/108] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:11 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:11.714105 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9c35e-7250-4338-9cc6-4ff595b2f916 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:11.755661 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:11.855134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41f9c35e-7250-4338-9cc6-4ff595b2f916 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:11.890442 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212e09f1-a696-475f-861f-9391ce837813 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:11.898901 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:11.899904 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d50175-4b5c-4131-943e-3985a4059097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:11.900724 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d50175-4b5c-4131-943e-3985a4059097 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:11.901004 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d50175-4b5c-4131-943e-3985a4059097 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:11.902648 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d50175-4b5c-4131-943e-3985a4059097 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:11.917290 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d50175-4b5c-4131-943e-3985a4059097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:11.928189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-d9d50175-4b5c-4131-943e-3985a4059097 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 16:53:11.928189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:11.928189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:11.928189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:11.928189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:11.928189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:11.928189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:11.928189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:11.928189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:11.928189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:11.928189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:11.928189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:11.928189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:11.928189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:11.928189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:11.928189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:11.929854 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:11.929854 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:11.929854 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:11.929854 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:11.929854 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:11.929854 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:11.929854 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:11.929854 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:11.929854 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:11.929854 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:11.929854 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:11.929854 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:11.929854 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:11.929854 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:11.929854 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:11.929854 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:11.931058 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:11.931058 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:11.931058 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:11.931058 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:11.931058 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:11.931058 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:11.931058 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:11.931058 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:11.931058 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:11.931058 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:11.931058 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 16:53:11.931058 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:11.931058 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 55/109] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:11.931058 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d50175-4b5c-4131-943e-3985a4059097 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:11.942935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31af5586-106a-41e9-96c5-562729773bbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:11.942935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af5586-106a-41e9-96c5-562729773bbc None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:11.942935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af5586-106a-41e9-96c5-562729773bbc None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:11.942935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af5586-106a-41e9-96c5-562729773bbc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:11.948163 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31af5586-106a-41e9-96c5-562729773bbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:11.957256 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-31af5586-106a-41e9-96c5-562729773bbc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:53:11.957256 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:11.957256 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:11.957256 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:11.957256 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:11.957256 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:11.957256 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:11.957256 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:11.957256 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:11.957256 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:11.957256 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:11.957256 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:11.957256 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:11.957256 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:11.957256 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:11.957256 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:11.959010 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:11.959010 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:11.959010 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:11.959010 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:11.959010 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:11.959010 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:11.959010 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:11.959010 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:11.959010 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:11.959010 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:11.959010 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:11.959010 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:11.959010 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:11.959010 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:11.959010 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:11.959010 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:11.960907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:11.960907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:11.960907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:11.960907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:11.960907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:11.960907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:11.960907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:11.960907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:11.960907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:11.960907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:11.960907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:53:11.960907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:11.960907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 55/110] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:53:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:11.965890 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31af5586-106a-41e9-96c5-562729773bbc None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:12.035135 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:12.055915 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d50175-4b5c-4131-943e-3985a4059097 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:12.093423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c8270-ed19-4243-8821-21dd9d44378c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:12.094104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c8270-ed19-4243-8821-21dd9d44378c None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:12.094389 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c8270-ed19-4243-8821-21dd9d44378c None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:12.094669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5c8270-ed19-4243-8821-21dd9d44378c None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:12.095472 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c8270-ed19-4243-8821-21dd9d44378c None admin] RBAC: Adding query filter params (name=member) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:12.095782 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c8270-ed19-4243-8821-21dd9d44378c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:12.097260 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5c8270-ed19-4243-8821-21dd9d44378c None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:12.100555 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31af5586-106a-41e9-96c5-562729773bbc None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:12.107235 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 56/111] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:12.108675 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c8270-ed19-4243-8821-21dd9d44378c None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:12.139736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13517e4-188e-485c-8494-d538230994a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:12.140612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13517e4-188e-485c-8494-d538230994a1 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:12.140910 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13517e4-188e-485c-8494-d538230994a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:12.141191 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13517e4-188e-485c-8494-d538230994a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:12.141967 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13517e4-188e-485c-8494-d538230994a1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:12.142337 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13517e4-188e-485c-8494-d538230994a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:12.143979 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13517e4-188e-485c-8494-d538230994a1 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:12.154739 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 56/112] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:12.155755 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13517e4-188e-485c-8494-d538230994a1 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:12.193521 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:12.243877 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5c8270-ed19-4243-8821-21dd9d44378c None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:12.287484 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672dd4e7-9c1b-49a0-8147-6d331011b3d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:12.288374 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672dd4e7-9c1b-49a0-8147-6d331011b3d6 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:12.288656 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672dd4e7-9c1b-49a0-8147-6d331011b3d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:12.289086 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672dd4e7-9c1b-49a0-8147-6d331011b3d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:12.289831 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672dd4e7-9c1b-49a0-8147-6d331011b3d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:12.290139 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672dd4e7-9c1b-49a0-8147-6d331011b3d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:12.291555 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672dd4e7-9c1b-49a0-8147-6d331011b3d6 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:12.300959 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 57/113] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:12.302032 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672dd4e7-9c1b-49a0-8147-6d331011b3d6 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:12.314301 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:12.376737 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13517e4-188e-485c-8494-d538230994a1 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:12.424128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd329470-1d37-493e-860b-38b29fbdcdb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:12.425821 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd329470-1d37-493e-860b-38b29fbdcdb3 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:12.426209 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd329470-1d37-493e-860b-38b29fbdcdb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:12.426760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd329470-1d37-493e-860b-38b29fbdcdb3 None admin] PATH_INFO: `/v3/users/99a4070a60cc4ae8a1981ac1c10668fe` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:12.428370 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd329470-1d37-493e-860b-38b29fbdcdb3 None admin] RBAC: Authorizing `identity:get_user(user_id=99a4070a60cc4ae8a1981ac1c10668fe)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:12.430891 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd329470-1d37-493e-860b-38b29fbdcdb3 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:12.433564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 57/114] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:12 2022] GET /identity/v3/users/99a4070a60cc4ae8a1981ac1c10668fe => generated 275 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:12.434547 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd329470-1d37-493e-860b-38b29fbdcdb3 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:12.435771 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672dd4e7-9c1b-49a0-8147-6d331011b3d6 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:12.480877 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323596cf-e860-485a-84ef-f91fd3e08486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:12.481943 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323596cf-e860-485a-84ef-f91fd3e08486 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:12.482227 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323596cf-e860-485a-84ef-f91fd3e08486 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:12.482498 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323596cf-e860-485a-84ef-f91fd3e08486 None admin] PATH_INFO: `/v3/users/0de58fd59284495d9eab8ab28e53612c` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:12.497255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323596cf-e860-485a-84ef-f91fd3e08486 None admin] RBAC: Authorizing `identity:get_user(user_id=0de58fd59284495d9eab8ab28e53612c)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:12.498375 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323596cf-e860-485a-84ef-f91fd3e08486 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:12.500123 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 58/115] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:12 2022] GET /identity/v3/users/0de58fd59284495d9eab8ab28e53612c => generated 289 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:12.501381 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7a100-96c4-44e8-9a9d-099a0cf58106 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:12.501654 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7a100-96c4-44e8-9a9d-099a0cf58106 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:12.501929 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a7a100-96c4-44e8-9a9d-099a0cf58106 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:12.507190 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 59/116] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:12.507190 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7a100-96c4-44e8-9a9d-099a0cf58106 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:12.550943 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:12.559248 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd329470-1d37-493e-860b-38b29fbdcdb3 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:12.574626 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:12.594832 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0e66ed-1742-4c5d-927b-d02cbac734dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:12.595747 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e66ed-1742-4c5d-927b-d02cbac734dd None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:12.596024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e66ed-1742-4c5d-927b-d02cbac734dd None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:12.596306 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0e66ed-1742-4c5d-927b-d02cbac734dd None admin] PATH_INFO: `/v3/users/d1134eaf564846f5990bd2b2934b7db7` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:12.597477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e66ed-1742-4c5d-927b-d02cbac734dd None admin] RBAC: Authorizing `identity:get_user(user_id=d1134eaf564846f5990bd2b2934b7db7)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:12.598977 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0e66ed-1742-4c5d-927b-d02cbac734dd None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:12.600682 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 58/117] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:12 2022] GET /identity/v3/users/d1134eaf564846f5990bd2b2934b7db7 => generated 247 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:12.605148 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96204f-d942-474a-90bc-8cc273b94a73 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:12.605148 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96204f-d942-474a-90bc-8cc273b94a73 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:12.605148 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d96204f-d942-474a-90bc-8cc273b94a73 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:12.605148 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 59/118] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:12.608562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96204f-d942-474a-90bc-8cc273b94a73 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:12.618265 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a7a100-96c4-44e8-9a9d-099a0cf58106 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:12.654083 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8485b0-3c6c-43e9-a306-674208e9d176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:12.654984 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8485b0-3c6c-43e9-a306-674208e9d176 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:12.655274 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8485b0-3c6c-43e9-a306-674208e9d176 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:12.655552 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8485b0-3c6c-43e9-a306-674208e9d176 None admin] PATH_INFO: `/v3/projects/f29295d0fb6141208e3a094c3be29944` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:12.656715 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8485b0-3c6c-43e9-a306-674208e9d176 None admin] RBAC: Authorizing `identity:get_project(project_id=f29295d0fb6141208e3a094c3be29944)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:12.658197 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8485b0-3c6c-43e9-a306-674208e9d176 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:12.659926 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 60/119] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:12 2022] GET /identity/v3/projects/f29295d0fb6141208e3a094c3be29944 => generated 298 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:12.661129 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa202dee-5926-4825-b7c5-38e3d0f4123d None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:12.661412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa202dee-5926-4825-b7c5-38e3d0f4123d None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:12.661684 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa202dee-5926-4825-b7c5-38e3d0f4123d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:12.677943 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-fa202dee-5926-4825-b7c5-38e3d0f4123d None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:12.679222 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-fa202dee-5926-4825-b7c5-38e3d0f4123d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:12.730400 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d96204f-d942-474a-90bc-8cc273b94a73 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:12.749946 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 61/120] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:12.751110 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa202dee-5926-4825-b7c5-38e3d0f4123d None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:12.775658 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883548dd-d11c-4f62-9b62-418e5c5cc8fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:12.776393 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883548dd-d11c-4f62-9b62-418e5c5cc8fc None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:12.776605 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883548dd-d11c-4f62-9b62-418e5c5cc8fc None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:12.776879 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883548dd-d11c-4f62-9b62-418e5c5cc8fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:12.809399 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:12.822070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:12.849509 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883548dd-d11c-4f62-9b62-418e5c5cc8fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:12.851743 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883548dd-d11c-4f62-9b62-418e5c5cc8fc None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:12.862428 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa202dee-5926-4825-b7c5-38e3d0f4123d None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:12.899075 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00f4e7-5c5f-4a9a-8e81-9ac2d7ceaada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:12.900205 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00f4e7-5c5f-4a9a-8e81-9ac2d7ceaada None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:12.900788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00f4e7-5c5f-4a9a-8e81-9ac2d7ceaada None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:12.900788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00f4e7-5c5f-4a9a-8e81-9ac2d7ceaada None admin] PATH_INFO: `/v3/projects/f29295d0fb6141208e3a094c3be29944` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:12.902769 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00f4e7-5c5f-4a9a-8e81-9ac2d7ceaada None admin] RBAC: Authorizing `identity:get_project(project_id=f29295d0fb6141208e3a094c3be29944)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:12.904256 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da00f4e7-5c5f-4a9a-8e81-9ac2d7ceaada None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:12.905946 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 62/121] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:12 2022] GET /identity/v3/projects/f29295d0fb6141208e3a094c3be29944 => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:12.907168 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90490770-3a60-4bb5-9df5-19c9fa8e469f None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:12.907453 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90490770-3a60-4bb5-9df5-19c9fa8e469f None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:12.907724 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90490770-3a60-4bb5-9df5-19c9fa8e469f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:12.924644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-90490770-3a60-4bb5-9df5-19c9fa8e469f None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:12.925801 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-90490770-3a60-4bb5-9df5-19c9fa8e469f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:12.969814 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 60/122] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:53:12.970751 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883548dd-d11c-4f62-9b62-418e5c5cc8fc None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:12.998871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 63/123] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:12.999980 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90490770-3a60-4bb5-9df5-19c9fa8e469f None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:13.048520 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:13.071158 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:13.087644 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883548dd-d11c-4f62-9b62-418e5c5cc8fc None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:13.115991 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90490770-3a60-4bb5-9df5-19c9fa8e469f None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:13.121875 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0ad3c-4d03-4f53-8a58-993b7bd8d155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:13.122772 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0ad3c-4d03-4f53-8a58-993b7bd8d155 None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:13.123096 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0ad3c-4d03-4f53-8a58-993b7bd8d155 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:13.123373 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f0ad3c-4d03-4f53-8a58-993b7bd8d155 None admin] PATH_INFO: `/v3/projects/f29295d0fb6141208e3a094c3be29944/users/99a4070a60cc4ae8a1981ac1c10668fe/roles/72cc26a1a66c4fc183b39fad4cb38431` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:13.125350 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f0ad3c-4d03-4f53-8a58-993b7bd8d155 None admin] RBAC: Authorizing `identity:create_grant(project_id=f29295d0fb6141208e3a094c3be29944, user_id=99a4070a60cc4ae8a1981ac1c10668fe, role_id=72cc26a1a66c4fc183b39fad4cb38431)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:13.126821 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f0ad3c-4d03-4f53-8a58-993b7bd8d155 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:13.143224 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 61/124] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:53:12 2022] PUT /identity/v3/projects/f29295d0fb6141208e3a094c3be29944/users/99a4070a60cc4ae8a1981ac1c10668fe/roles/72cc26a1a66c4fc183b39fad4cb38431 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:53:13.144022 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0ad3c-4d03-4f53-8a58-993b7bd8d155 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:13.169983 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61372e76-3d70-4c1d-b48c-a7973b6d21e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:13.171420 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61372e76-3d70-4c1d-b48c-a7973b6d21e6 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:13.172029 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61372e76-3d70-4c1d-b48c-a7973b6d21e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:13.172503 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61372e76-3d70-4c1d-b48c-a7973b6d21e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:13.186243 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61372e76-3d70-4c1d-b48c-a7973b6d21e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:13.195404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-61372e76-3d70-4c1d-b48c-a7973b6d21e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:13.195404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:13.195404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:13.195404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:13.195404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:13.195404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:13.195404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:13.195404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:13.195404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:13.195404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:13.195404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:13.195404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:13.195404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:13.195404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:13.195404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:13.195404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:13.196348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:13.196348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:13.196348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:13.196348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:13.196348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:13.196348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:13.196348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:13.196348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:13.196348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:13.196348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:13.196348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:13.196348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:13.196348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:13.196348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:13.196348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:13.196348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:13.197104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:13.197104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:13.197104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:13.197104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:13.197104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:13.197104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:13.197104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:13.197104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:13.197104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:13.197104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:13.197104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:13.197104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:13.199973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 64/125] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:13.206046 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61372e76-3d70-4c1d-b48c-a7973b6d21e6 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:13.283758 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f0ad3c-4d03-4f53-8a58-993b7bd8d155 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:13.316186 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:13.331105 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a416ef-aef6-42bd-aaf6-99165d5dc93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:13.332675 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a416ef-aef6-42bd-aaf6-99165d5dc93d None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:13.333563 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a416ef-aef6-42bd-aaf6-99165d5dc93d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:13.334246 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a416ef-aef6-42bd-aaf6-99165d5dc93d None admin] PATH_INFO: `/v3/projects/f29295d0fb6141208e3a094c3be29944/users/d1134eaf564846f5990bd2b2934b7db7/roles/17bbe7cc1c3641509de5e9dca8534102` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:13.337308 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a416ef-aef6-42bd-aaf6-99165d5dc93d None admin] RBAC: Authorizing `identity:create_grant(project_id=f29295d0fb6141208e3a094c3be29944, user_id=d1134eaf564846f5990bd2b2934b7db7, role_id=17bbe7cc1c3641509de5e9dca8534102)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:13.339547 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a416ef-aef6-42bd-aaf6-99165d5dc93d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:13.348067 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61372e76-3d70-4c1d-b48c-a7973b6d21e6 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:13.360466 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 62/126] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:53:13 2022] PUT /identity/v3/projects/f29295d0fb6141208e3a094c3be29944/users/d1134eaf564846f5990bd2b2934b7db7/roles/17bbe7cc1c3641509de5e9dca8534102 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:53:13.361839 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a416ef-aef6-42bd-aaf6-99165d5dc93d None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:13.385513 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:13.403120 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70745a5-122b-484e-a35d-2f6e42653fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:13.404622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70745a5-122b-484e-a35d-2f6e42653fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:13.405204 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70745a5-122b-484e-a35d-2f6e42653fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:13.405776 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70745a5-122b-484e-a35d-2f6e42653fc9 None admin] PATH_INFO: `/v3/projects/f29295d0fb6141208e3a094c3be29944` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:13.407577 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70745a5-122b-484e-a35d-2f6e42653fc9 None admin] RBAC: Authorizing `identity:get_project(project_id=f29295d0fb6141208e3a094c3be29944)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:13.409440 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70745a5-122b-484e-a35d-2f6e42653fc9 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:13.412030 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 65/127] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:13 2022] GET /identity/v3/projects/f29295d0fb6141208e3a094c3be29944 => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:13.418671 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70745a5-122b-484e-a35d-2f6e42653fc9 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:13.519998 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a416ef-aef6-42bd-aaf6-99165d5dc93d None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:13.549145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70745a5-122b-484e-a35d-2f6e42653fc9 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:13.570116 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a504887-c424-449f-b3ab-ee5ec1aa3251 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:13.571466 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a504887-c424-449f-b3ab-ee5ec1aa3251 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:13.572945 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a504887-c424-449f-b3ab-ee5ec1aa3251 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:13.573407 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a504887-c424-449f-b3ab-ee5ec1aa3251 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:13.588885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a504887-c424-449f-b3ab-ee5ec1aa3251 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:13.598518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-3a504887-c424-449f-b3ab-ee5ec1aa3251 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:53:13.598518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:13.598518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:13.598518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:13.598518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:13.598518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:13.598518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:13.598518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:13.598518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:13.598518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:13.598518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:13.598518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:13.598518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:13.598518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:13.598518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:13.598518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:13.599648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:13.599648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:13.599648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:13.599648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:13.599648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:13.599648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:13.599648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:13.599648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:13.599648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:13.599648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:13.599648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:13.599648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:13.599648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:13.599648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:13.599648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:13.599648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:13.600681 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:13.600681 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:13.600681 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:13.600681 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:13.600681 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:13.600681 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:13.600681 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:13.600681 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:13.600681 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:13.600681 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:13.600681 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:53:13.600681 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:13.601255 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 63/128] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 23 16:53:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:13.603296 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a504887-c424-449f-b3ab-ee5ec1aa3251 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:13.616669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19873569-7d6c-4ecc-802c-38bf7a3b1e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:13.617636 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19873569-7d6c-4ecc-802c-38bf7a3b1e87 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:13.618053 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19873569-7d6c-4ecc-802c-38bf7a3b1e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:13.618474 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19873569-7d6c-4ecc-802c-38bf7a3b1e87 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:13.619408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19873569-7d6c-4ecc-802c-38bf7a3b1e87 None admin] RBAC: Adding query filter params (name=member) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:13.619863 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19873569-7d6c-4ecc-802c-38bf7a3b1e87 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:13.621454 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19873569-7d6c-4ecc-802c-38bf7a3b1e87 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:13.642323 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 66/129] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:13.643498 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19873569-7d6c-4ecc-802c-38bf7a3b1e87 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:13.666939 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:13.731629 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a504887-c424-449f-b3ab-ee5ec1aa3251 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:13.733316 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:13.780091 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0ef17-bb99-44f3-a8e1-d5bc6a115c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:13.781209 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0ef17-bb99-44f3-a8e1-d5bc6a115c1d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:13.781597 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0ef17-bb99-44f3-a8e1-d5bc6a115c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:13.782404 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a0ef17-bb99-44f3-a8e1-d5bc6a115c1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:13.783645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a0ef17-bb99-44f3-a8e1-d5bc6a115c1d None admin] RBAC: Adding query filter params (role.id=9d4694dd70b94d68acc78c53ca498bbf, scope.project.id=f29295d0fb6141208e3a094c3be29944, user.id=0de58fd59284495d9eab8ab28e53612c) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:13.784067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a0ef17-bb99-44f3-a8e1-d5bc6a115c1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:13.786637 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a0ef17-bb99-44f3-a8e1-d5bc6a115c1d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:13.788587 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19873569-7d6c-4ecc-802c-38bf7a3b1e87 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:13.798962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 64/130] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:53:13 2022] GET /identity/v3/role_assignments?user.id=0de58fd59284495d9eab8ab28e53612c&scope.project.id=f29295d0fb6141208e3a094c3be29944&role.id=9d4694dd70b94d68acc78c53ca498bbf => generated 259 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:13.798962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0ef17-bb99-44f3-a8e1-d5bc6a115c1d None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:13.876314 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc109b33-babd-4432-976d-b738cc350615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:13.877405 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc109b33-babd-4432-976d-b738cc350615 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:13.877899 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc109b33-babd-4432-976d-b738cc350615 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:13.878414 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc109b33-babd-4432-976d-b738cc350615 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:13.879452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc109b33-babd-4432-976d-b738cc350615 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:13.879974 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc109b33-babd-4432-976d-b738cc350615 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:13.881693 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc109b33-babd-4432-976d-b738cc350615 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:13.901251 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 67/131] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:53:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:13.924911 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc109b33-babd-4432-976d-b738cc350615 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:13.925118 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a0ef17-bb99-44f3-a8e1-d5bc6a115c1d None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:13.937273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:13.967916 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9581efc7-ae96-4464-abc6-ba7833ddb528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:13.969202 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581efc7-ae96-4464-abc6-ba7833ddb528 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:13.969548 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581efc7-ae96-4464-abc6-ba7833ddb528 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:13.969833 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9581efc7-ae96-4464-abc6-ba7833ddb528 None admin] PATH_INFO: `/v3/users/99a4070a60cc4ae8a1981ac1c10668fe` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:13.971153 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9581efc7-ae96-4464-abc6-ba7833ddb528 None admin] RBAC: Authorizing `identity:get_user(user_id=99a4070a60cc4ae8a1981ac1c10668fe)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:13.972656 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9581efc7-ae96-4464-abc6-ba7833ddb528 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:13.975132 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 65/132] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:13 2022] GET /identity/v3/users/99a4070a60cc4ae8a1981ac1c10668fe => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:13.981635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9581efc7-ae96-4464-abc6-ba7833ddb528 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:14.105293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9581efc7-ae96-4464-abc6-ba7833ddb528 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:14.143413 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef307f-5a9a-4d7e-a89c-367b8a9cf232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:14.144280 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef307f-5a9a-4d7e-a89c-367b8a9cf232 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:14.144564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef307f-5a9a-4d7e-a89c-367b8a9cf232 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:14.144848 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ef307f-5a9a-4d7e-a89c-367b8a9cf232 None admin] PATH_INFO: `/v3/projects/15bd2384217841e5bc06a9047dbfb59d` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:14.146096 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef307f-5a9a-4d7e-a89c-367b8a9cf232 None admin] RBAC: Authorizing `identity:get_project(project_id=15bd2384217841e5bc06a9047dbfb59d)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:14.147661 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ef307f-5a9a-4d7e-a89c-367b8a9cf232 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:14.149884 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 66/133] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:13 2022] GET /identity/v3/projects/15bd2384217841e5bc06a9047dbfb59d => generated 312 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:14.156172 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef307f-5a9a-4d7e-a89c-367b8a9cf232 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:14.193178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc109b33-babd-4432-976d-b738cc350615 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:14.214692 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:14.229644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:14.265302 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3430f3-9356-4ee5-a156-9d559d603b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:14.266197 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3430f3-9356-4ee5-a156-9d559d603b92 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:14.266484 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3430f3-9356-4ee5-a156-9d559d603b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:14.266760 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3430f3-9356-4ee5-a156-9d559d603b92 None admin] PATH_INFO: `/v3/users/99a4070a60cc4ae8a1981ac1c10668fe` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:14.268003 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3430f3-9356-4ee5-a156-9d559d603b92 None admin] RBAC: Authorizing `identity:get_user(user_id=99a4070a60cc4ae8a1981ac1c10668fe)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:14.269586 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3430f3-9356-4ee5-a156-9d559d603b92 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:14.271313 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 68/134] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:13 2022] GET /identity/v3/users/99a4070a60cc4ae8a1981ac1c10668fe => generated 275 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:14.277439 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3430f3-9356-4ee5-a156-9d559d603b92 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:14.372154 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ef307f-5a9a-4d7e-a89c-367b8a9cf232 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:14.383707 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3430f3-9356-4ee5-a156-9d559d603b92 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:14.441598 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43e865-ed27-4463-b2f5-3f02f8a017cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:14.442430 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43e865-ed27-4463-b2f5-3f02f8a017cb None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:14.442714 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43e865-ed27-4463-b2f5-3f02f8a017cb None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:14.443017 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43e865-ed27-4463-b2f5-3f02f8a017cb None admin] PATH_INFO: `/v3/projects/15bd2384217841e5bc06a9047dbfb59d/users/99a4070a60cc4ae8a1981ac1c10668fe/roles/72cc26a1a66c4fc183b39fad4cb38431` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:14.445762 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43e865-ed27-4463-b2f5-3f02f8a017cb None admin] RBAC: Authorizing `identity:create_grant(project_id=15bd2384217841e5bc06a9047dbfb59d, user_id=99a4070a60cc4ae8a1981ac1c10668fe, role_id=72cc26a1a66c4fc183b39fad4cb38431)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:14.447260 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43e865-ed27-4463-b2f5-3f02f8a017cb None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:14.453464 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626f6f55-ba19-4275-8eae-f3609e074e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:14.454579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f6f55-ba19-4275-8eae-f3609e074e80 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:14.454967 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f6f55-ba19-4275-8eae-f3609e074e80 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:14.455322 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626f6f55-ba19-4275-8eae-f3609e074e80 None admin] PATH_INFO: `/v3/projects/f29295d0fb6141208e3a094c3be29944` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:14.456798 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f6f55-ba19-4275-8eae-f3609e074e80 None admin] RBAC: Authorizing `identity:get_project(project_id=f29295d0fb6141208e3a094c3be29944)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:14.458253 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626f6f55-ba19-4275-8eae-f3609e074e80 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:14.460431 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 69/135] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:14 2022] GET /identity/v3/projects/f29295d0fb6141208e3a094c3be29944 => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:14.461655 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9ff62-be2c-4000-8bfd-4becc6327477 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:14.461938 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9ff62-be2c-4000-8bfd-4becc6327477 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:14.462216 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9ff62-be2c-4000-8bfd-4becc6327477 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:14.463453 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 70/136] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:14 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:14.473352 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 67/137] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:53:14 2022] PUT /identity/v3/projects/15bd2384217841e5bc06a9047dbfb59d/users/99a4070a60cc4ae8a1981ac1c10668fe/roles/72cc26a1a66c4fc183b39fad4cb38431 => generated 0 bytes in 318 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:53:14.474173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43e865-ed27-4463-b2f5-3f02f8a017cb None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:14.475511 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf97c3-be3b-4aec-9bf3-3408d745549a None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:14.475807 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf97c3-be3b-4aec-9bf3-3408d745549a None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:14.476087 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaf97c3-be3b-4aec-9bf3-3408d745549a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:14.508645 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-fcaf97c3-be3b-4aec-9bf3-3408d745549a None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:14.511766 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-fcaf97c3-be3b-4aec-9bf3-3408d745549a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:14.591555 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:14.633327 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 71/138] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:14.666229 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43e865-ed27-4463-b2f5-3f02f8a017cb None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:14.670436 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:14.675847 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf97c3-be3b-4aec-9bf3-3408d745549a None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:14.767159 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b0baf-5478-4781-af29-896b3b29b7ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:14.767985 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b0baf-5478-4781-af29-896b3b29b7ce None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:14.768266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b0baf-5478-4781-af29-896b3b29b7ce None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:14.768595 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-636b0baf-5478-4781-af29-896b3b29b7ce None admin] PATH_INFO: `/v3/projects/f29295d0fb6141208e3a094c3be29944/users/99a4070a60cc4ae8a1981ac1c10668fe/roles/6c30aca17fad4c4fa40b9461c5bda291` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:14.775968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636b0baf-5478-4781-af29-896b3b29b7ce None admin] RBAC: Authorizing `identity:create_grant(project_id=f29295d0fb6141208e3a094c3be29944, user_id=99a4070a60cc4ae8a1981ac1c10668fe, role_id=6c30aca17fad4c4fa40b9461c5bda291)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:14.777443 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-636b0baf-5478-4781-af29-896b3b29b7ce None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:14.796205 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 68/139] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:53:14 2022] PUT /identity/v3/projects/f29295d0fb6141208e3a094c3be29944/users/99a4070a60cc4ae8a1981ac1c10668fe/roles/6c30aca17fad4c4fa40b9461c5bda291 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:53:14.867235 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaf97c3-be3b-4aec-9bf3-3408d745549a None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:14.910310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750cf1a5-4e98-4438-93d2-604ffc3a40b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:14.911144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750cf1a5-4e98-4438-93d2-604ffc3a40b7 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:14.911422 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750cf1a5-4e98-4438-93d2-604ffc3a40b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:14.911694 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750cf1a5-4e98-4438-93d2-604ffc3a40b7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:14.915876 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750cf1a5-4e98-4438-93d2-604ffc3a40b7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:14.917398 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750cf1a5-4e98-4438-93d2-604ffc3a40b7 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:14.919447 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 72/140] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:53:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:14.924688 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b0baf-5478-4781-af29-896b3b29b7ce None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:15.041161 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-636b0baf-5478-4781-af29-896b3b29b7ce None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:15.077406 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49498c00-8bd6-404a-ba50-ae40a6326eef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:15.078090 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49498c00-8bd6-404a-ba50-ae40a6326eef None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:15.078371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49498c00-8bd6-404a-ba50-ae40a6326eef None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:15.078644 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49498c00-8bd6-404a-ba50-ae40a6326eef None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:15.079403 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49498c00-8bd6-404a-ba50-ae40a6326eef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:15.080871 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49498c00-8bd6-404a-ba50-ae40a6326eef None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:15.106222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 69/141] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 23 16:53:14 2022] POST /identity/v3/users => generated 283 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:53:16.451665 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe2ae8-8e58-4029-abd6-6779bc487cb3 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:16.452206 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe2ae8-8e58-4029-abd6-6779bc487cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:16.452206 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fe2ae8-8e58-4029-abd6-6779bc487cb3 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:16.453778 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 73/142] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:16.458898 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917e66b-f80f-4277-93a0-30bff27f2515 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:16.459386 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917e66b-f80f-4277-93a0-30bff27f2515 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:16.459622 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b917e66b-f80f-4277-93a0-30bff27f2515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:16.492562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-b917e66b-f80f-4277-93a0-30bff27f2515 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:16.493765 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-b917e66b-f80f-4277-93a0-30bff27f2515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:16.512757 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:16.519764 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c631de-fefa-4ed2-ae68-07c84b0ed9f9 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:16.520049 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c631de-fefa-4ed2-ae68-07c84b0ed9f9 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:16.520326 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c631de-fefa-4ed2-ae68-07c84b0ed9f9 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:16.521707 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 74/143] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:53:16.528529 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66954817-02c8-4024-b10a-136027c02a90 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:16.528859 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66954817-02c8-4024-b10a-136027c02a90 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:16.529118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66954817-02c8-4024-b10a-136027c02a90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:16.554554 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-66954817-02c8-4024-b10a-136027c02a90 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:16.555775 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-66954817-02c8-4024-b10a-136027c02a90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:16.632445 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 70/144] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:16.671483 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:16.672915 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917e66b-f80f-4277-93a0-30bff27f2515 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:16.717800 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 75/145] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 16:53:16.761183 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66954817-02c8-4024-b10a-136027c02a90 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:16.842896 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b917e66b-f80f-4277-93a0-30bff27f2515 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:16.907186 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef1c710-d928-42c2-8c9c-b3ee8a09aa56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:16.908004 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1c710-d928-42c2-8c9c-b3ee8a09aa56 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:16.908280 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1c710-d928-42c2-8c9c-b3ee8a09aa56 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:16.908549 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef1c710-d928-42c2-8c9c-b3ee8a09aa56 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:16.928482 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef1c710-d928-42c2-8c9c-b3ee8a09aa56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:16.940444 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-fef1c710-d928-42c2-8c9c-b3ee8a09aa56 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 16:53:16.940444 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:16.940444 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:16.940444 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:16.940444 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:16.940444 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:16.940444 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:16.940444 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:16.940444 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:16.940444 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:16.940444 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:16.940444 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:16.940444 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:16.940444 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:16.940444 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:16.940444 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:16.941385 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:16.941385 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:16.941385 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:16.941385 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:16.941385 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:16.941385 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:16.941385 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:16.941385 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:16.941385 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:16.941385 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:16.941385 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:16.941385 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:16.941385 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:16.941385 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:16.941385 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:16.941385 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:16.942193 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:16.942193 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:16.942193 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:16.942193 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:16.942193 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:16.942193 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:16.942193 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:16.942193 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:16.942193 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:16.942193 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:16.942193 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 16:53:16.942193 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:16.942193 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 71/146] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:53:16.943301 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff30ef-019a-4281-adcf-b739d28bc995 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:16.944542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff30ef-019a-4281-adcf-b739d28bc995 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:16.944542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff30ef-019a-4281-adcf-b739d28bc995 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:16.945391 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 72/147] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:16.950632 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff30ef-019a-4281-adcf-b739d28bc995 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:16.971365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:16.977574 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66954817-02c8-4024-b10a-136027c02a90 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:17.083628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99c0e04-b773-4ed6-9c0f-7feed893eacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:17.084482 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c0e04-b773-4ed6-9c0f-7feed893eacf None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:17.085402 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c0e04-b773-4ed6-9c0f-7feed893eacf None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:17.085684 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99c0e04-b773-4ed6-9c0f-7feed893eacf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:17.115865 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99c0e04-b773-4ed6-9c0f-7feed893eacf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:17.124366 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-a99c0e04-b773-4ed6-9c0f-7feed893eacf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:53:17.124366 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:17.124366 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:17.124366 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:17.124366 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:17.124366 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:17.124366 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:17.124366 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:17.124366 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:17.124366 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:17.124366 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:17.124366 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:17.124366 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:17.124366 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:17.124366 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:17.124366 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:17.125221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:17.125221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:17.125221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:17.125221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:17.125221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:17.125221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:17.125221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:17.125221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:17.125221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:17.125221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:17.125221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:17.125221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:17.125221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:17.125221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:17.125221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:17.125221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:17.125982 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:17.125982 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:17.125982 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:17.125982 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:17.125982 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:17.125982 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:17.125982 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:17.125982 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:17.125982 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:17.125982 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:17.125982 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:53:17.125982 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:17.125982 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 76/148] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:53:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 365 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:53:17.127368 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d918136-491f-4451-b23a-1745ce2923de None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:17.135184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d918136-491f-4451-b23a-1745ce2923de None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:17.135427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff30ef-019a-4281-adcf-b739d28bc995 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:17.135524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d918136-491f-4451-b23a-1745ce2923de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:17.167922 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-2d918136-491f-4451-b23a-1745ce2923de None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:17.169101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-2d918136-491f-4451-b23a-1745ce2923de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:17.210426 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a87ad-aebe-4fb9-b89d-60b733d70c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:17.211135 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a87ad-aebe-4fb9-b89d-60b733d70c40 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:17.211422 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a87ad-aebe-4fb9-b89d-60b733d70c40 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:17.211703 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a87ad-aebe-4fb9-b89d-60b733d70c40 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:17.212467 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a87ad-aebe-4fb9-b89d-60b733d70c40 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:17.212776 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a87ad-aebe-4fb9-b89d-60b733d70c40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:17.214211 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a87ad-aebe-4fb9-b89d-60b733d70c40 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:17.231573 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 73/149] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:16 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:17.232288 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a87ad-aebe-4fb9-b89d-60b733d70c40 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:17.251929 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:17.340288 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 77/150] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:17.340907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:17.341170 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d918136-491f-4451-b23a-1745ce2923de None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:17.399641 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a87ad-aebe-4fb9-b89d-60b733d70c40 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:17.465918 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d7fca-8e73-4a59-9dff-da28a23e0644 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:17.465918 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d7fca-8e73-4a59-9dff-da28a23e0644 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:17.465918 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d7fca-8e73-4a59-9dff-da28a23e0644 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:17.465918 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109d7fca-8e73-4a59-9dff-da28a23e0644 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:17.466703 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d7fca-8e73-4a59-9dff-da28a23e0644 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:17.466703 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d7fca-8e73-4a59-9dff-da28a23e0644 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:17.467175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109d7fca-8e73-4a59-9dff-da28a23e0644 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:17.482084 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 74/151] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:17.482900 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d7fca-8e73-4a59-9dff-da28a23e0644 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:17.542513 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d918136-491f-4451-b23a-1745ce2923de None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:17.613826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ce0c9-bedf-4566-8acb-90936b017739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:17.614742 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ce0c9-bedf-4566-8acb-90936b017739 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:17.615056 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ce0c9-bedf-4566-8acb-90936b017739 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:17.615337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4ce0c9-bedf-4566-8acb-90936b017739 None admin] PATH_INFO: `/v3/users/0de58fd59284495d9eab8ab28e53612c` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:17.616899 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ce0c9-bedf-4566-8acb-90936b017739 None admin] RBAC: Authorizing `identity:get_user(user_id=0de58fd59284495d9eab8ab28e53612c)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:17.618373 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4ce0c9-bedf-4566-8acb-90936b017739 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:17.620225 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 78/152] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:17 2022] GET /identity/v3/users/0de58fd59284495d9eab8ab28e53612c => generated 289 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:17.622072 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ce0c9-bedf-4566-8acb-90936b017739 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:17.640031 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109d7fca-8e73-4a59-9dff-da28a23e0644 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:17.667300 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:17.715743 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c273e-13c2-4244-bb33-2967d5b1a0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:17.716569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c273e-13c2-4244-bb33-2967d5b1a0cf None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:17.716848 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c273e-13c2-4244-bb33-2967d5b1a0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:17.717116 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383c273e-13c2-4244-bb33-2967d5b1a0cf None admin] PATH_INFO: `/v3/roles/member` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:17.731264 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:17.744573 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383c273e-13c2-4244-bb33-2967d5b1a0cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:17.756385 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4ce0c9-bedf-4566-8acb-90936b017739 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:17.758513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-383c273e-13c2-4244-bb33-2967d5b1a0cf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:17.758513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:17.758513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:17.758513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:17.758513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:17.758513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:17.758513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:17.758513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:17.758513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:17.758513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:17.758513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:17.758513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:17.758513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:17.758513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:17.758513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:17.758513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:17.759885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:17.759885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:17.759885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:17.759885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:17.759885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:17.759885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:17.759885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:17.759885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:17.759885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:17.759885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:17.759885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:17.759885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:17.759885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:17.759885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:17.759885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:17.759885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:17.760823 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:17.760823 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:17.760823 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:17.760823 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:17.760823 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:17.760823 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:17.760823 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:17.760823 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:17.760823 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:17.760823 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:17.760823 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:17.760823 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:17.760823 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 75/153] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:17.761988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c273e-13c2-4244-bb33-2967d5b1a0cf None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:17.796264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a68478e-f116-497c-b330-ec76588d897c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:17.797171 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68478e-f116-497c-b330-ec76588d897c None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:17.797460 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68478e-f116-497c-b330-ec76588d897c None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:17.797744 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a68478e-f116-497c-b330-ec76588d897c None admin] PATH_INFO: `/v3/users/d1134eaf564846f5990bd2b2934b7db7` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:17.799052 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a68478e-f116-497c-b330-ec76588d897c None admin] RBAC: Authorizing `identity:get_user(user_id=d1134eaf564846f5990bd2b2934b7db7)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:17.800563 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a68478e-f116-497c-b330-ec76588d897c None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:17.802592 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 79/154] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:17 2022] GET /identity/v3/users/d1134eaf564846f5990bd2b2934b7db7 => generated 247 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:17.807749 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a68478e-f116-497c-b330-ec76588d897c None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:17.922569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383c273e-13c2-4244-bb33-2967d5b1a0cf None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:17.980756 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e726cd-89b6-4554-8578-f6ebd5e80a54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:17.981641 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e726cd-89b6-4554-8578-f6ebd5e80a54 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:17.981908 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e726cd-89b6-4554-8578-f6ebd5e80a54 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:17.982179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e726cd-89b6-4554-8578-f6ebd5e80a54 None admin] PATH_INFO: `/v3/projects/f29295d0fb6141208e3a094c3be29944` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:17.986163 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a68478e-f116-497c-b330-ec76588d897c None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:17.987665 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e726cd-89b6-4554-8578-f6ebd5e80a54 None admin] RBAC: Authorizing `identity:get_project(project_id=f29295d0fb6141208e3a094c3be29944)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:17.989168 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e726cd-89b6-4554-8578-f6ebd5e80a54 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:17.991581 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 76/155] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:17 2022] GET /identity/v3/projects/f29295d0fb6141208e3a094c3be29944 => generated 298 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:17.992094 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e726cd-89b6-4554-8578-f6ebd5e80a54 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:18.023420 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:18.035805 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ebc77-ca3f-4b0a-a2a1-3b9de789cd4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:18.036514 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ebc77-ca3f-4b0a-a2a1-3b9de789cd4f None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:18.036803 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ebc77-ca3f-4b0a-a2a1-3b9de789cd4f None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:18.037088 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1ebc77-ca3f-4b0a-a2a1-3b9de789cd4f None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:18.038459 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1ebc77-ca3f-4b0a-a2a1-3b9de789cd4f None admin] RBAC: Adding query filter params (name=member) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:18.038459 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1ebc77-ca3f-4b0a-a2a1-3b9de789cd4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:18.040000 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1ebc77-ca3f-4b0a-a2a1-3b9de789cd4f None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:18.049443 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 80/156] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:18.050377 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ebc77-ca3f-4b0a-a2a1-3b9de789cd4f None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:18.087383 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:18.152752 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e726cd-89b6-4554-8578-f6ebd5e80a54 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:18.160848 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1ebc77-ca3f-4b0a-a2a1-3b9de789cd4f None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:18.199853 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324ab57-4f94-4dcf-97f0-f2b51fccebea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:18.200888 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324ab57-4f94-4dcf-97f0-f2b51fccebea None admin] REQUEST_METHOD: `PUT` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:18.201171 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324ab57-4f94-4dcf-97f0-f2b51fccebea None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:18.201448 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324ab57-4f94-4dcf-97f0-f2b51fccebea None admin] PATH_INFO: `/v3/projects/f29295d0fb6141208e3a094c3be29944/users/0de58fd59284495d9eab8ab28e53612c/roles/9d4694dd70b94d68acc78c53ca498bbf` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:18.207657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534ce9e-b5af-4bdf-91e3-844e827500db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:18.207954 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324ab57-4f94-4dcf-97f0-f2b51fccebea None admin] RBAC: Authorizing `identity:create_grant(project_id=f29295d0fb6141208e3a094c3be29944, user_id=0de58fd59284495d9eab8ab28e53612c, role_id=9d4694dd70b94d68acc78c53ca498bbf)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:18.208330 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534ce9e-b5af-4bdf-91e3-844e827500db None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:18.208610 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534ce9e-b5af-4bdf-91e3-844e827500db None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:18.208881 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534ce9e-b5af-4bdf-91e3-844e827500db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:18.209512 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324ab57-4f94-4dcf-97f0-f2b51fccebea None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:18.228813 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 81/157] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:53:18 2022] PUT /identity/v3/projects/f29295d0fb6141208e3a094c3be29944/users/0de58fd59284495d9eab8ab28e53612c/roles/9d4694dd70b94d68acc78c53ca498bbf => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:53:18.229567 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324ab57-4f94-4dcf-97f0-f2b51fccebea None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:18.298486 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:18.329287 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534ce9e-b5af-4bdf-91e3-844e827500db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:18.330796 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534ce9e-b5af-4bdf-91e3-844e827500db None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:18.404846 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324ab57-4f94-4dcf-97f0-f2b51fccebea None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:18.447300 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:18.475797 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89daadb5-4385-497e-b7c3-1b30002fd432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:18.476719 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89daadb5-4385-497e-b7c3-1b30002fd432 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:18.477007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89daadb5-4385-497e-b7c3-1b30002fd432 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:18.477288 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89daadb5-4385-497e-b7c3-1b30002fd432 None admin] PATH_INFO: `/v3/users/99a4070a60cc4ae8a1981ac1c10668fe` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:18.483690 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89daadb5-4385-497e-b7c3-1b30002fd432 None admin] RBAC: Authorizing `identity:get_user(user_id=99a4070a60cc4ae8a1981ac1c10668fe)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:18.485176 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89daadb5-4385-497e-b7c3-1b30002fd432 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:18.487399 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 82/158] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:18 2022] GET /identity/v3/users/99a4070a60cc4ae8a1981ac1c10668fe => generated 275 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:18.487691 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 77/159] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:53:18.493089 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89daadb5-4385-497e-b7c3-1b30002fd432 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:18.500393 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534ce9e-b5af-4bdf-91e3-844e827500db None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:18.638292 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:18.650098 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534ce9e-b5af-4bdf-91e3-844e827500db None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:18.651250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89daadb5-4385-497e-b7c3-1b30002fd432 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:18.711733 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1e8837-d2ef-4d42-b089-336d2db02c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:18.712631 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e8837-d2ef-4d42-b089-336d2db02c9d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:18.712915 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e8837-d2ef-4d42-b089-336d2db02c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:18.713178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1e8837-d2ef-4d42-b089-336d2db02c9d None admin] PATH_INFO: `/v3/projects/f29295d0fb6141208e3a094c3be29944` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:18.714451 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e8837-d2ef-4d42-b089-336d2db02c9d None admin] RBAC: Authorizing `identity:get_project(project_id=f29295d0fb6141208e3a094c3be29944)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:18.715942 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1e8837-d2ef-4d42-b089-336d2db02c9d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:18.717626 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 78/160] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:18 2022] GET /identity/v3/projects/f29295d0fb6141208e3a094c3be29944 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:18.718392 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f1820-05ec-45ec-b47d-c5785b44fd45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:18.718783 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4eb117-bd19-4566-9817-3147880a03c3 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:18.719084 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4eb117-bd19-4566-9817-3147880a03c3 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:18.719173 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f1820-05ec-45ec-b47d-c5785b44fd45 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:18.719352 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4eb117-bd19-4566-9817-3147880a03c3 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:18.719452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f1820-05ec-45ec-b47d-c5785b44fd45 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:18.719634 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105f1820-05ec-45ec-b47d-c5785b44fd45 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:18.720664 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 79/161] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:18.726405 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365c163-bd94-47f7-b38b-ec87292dc2c1 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:18.726687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365c163-bd94-47f7-b38b-ec87292dc2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:18.726966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f365c163-bd94-47f7-b38b-ec87292dc2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:18.756782 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-f365c163-bd94-47f7-b38b-ec87292dc2c1 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:18.757840 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f365c163-bd94-47f7-b38b-ec87292dc2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:18.787284 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:18.797666 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f1820-05ec-45ec-b47d-c5785b44fd45 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:18.799698 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105f1820-05ec-45ec-b47d-c5785b44fd45 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:18.909072 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 83/162] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:53:18.910219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f1820-05ec-45ec-b47d-c5785b44fd45 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:18.914079 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:18.941048 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 80/163] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 16:53:18.941758 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365c163-bd94-47f7-b38b-ec87292dc2c1 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:19.038379 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105f1820-05ec-45ec-b47d-c5785b44fd45 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:19.075361 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f365c163-bd94-47f7-b38b-ec87292dc2c1 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:19.115700 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4512610-0ef3-449b-aebd-f8adf0a1c33e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:19.116440 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4512610-0ef3-449b-aebd-f8adf0a1c33e None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:19.116655 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4512610-0ef3-449b-aebd-f8adf0a1c33e None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:19.116936 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4512610-0ef3-449b-aebd-f8adf0a1c33e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:19.117781 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4512610-0ef3-449b-aebd-f8adf0a1c33e None admin] RBAC: Adding query filter params (role.id=17bbe7cc1c3641509de5e9dca8534102, scope.project.id=f29295d0fb6141208e3a094c3be29944, user.id=d1134eaf564846f5990bd2b2934b7db7) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:19.118097 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4512610-0ef3-449b-aebd-f8adf0a1c33e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:19.119611 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4512610-0ef3-449b-aebd-f8adf0a1c33e None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:19.122221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3361ba6-517b-44cb-801c-49ea9b2bbcf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:19.123120 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3361ba6-517b-44cb-801c-49ea9b2bbcf9 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:19.123394 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3361ba6-517b-44cb-801c-49ea9b2bbcf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:19.123661 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3361ba6-517b-44cb-801c-49ea9b2bbcf9 None admin] PATH_INFO: `/v3/projects/f29295d0fb6141208e3a094c3be29944` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:19.124841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3361ba6-517b-44cb-801c-49ea9b2bbcf9 None admin] RBAC: Authorizing `identity:get_project(project_id=f29295d0fb6141208e3a094c3be29944)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:19.126157 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 84/164] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:53:18 2022] GET /identity/v3/role_assignments?user.id=d1134eaf564846f5990bd2b2934b7db7&scope.project.id=f29295d0fb6141208e3a094c3be29944&role.id=17bbe7cc1c3641509de5e9dca8534102 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:19.126274 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3361ba6-517b-44cb-801c-49ea9b2bbcf9 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:19.128074 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 81/165] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:18 2022] GET /identity/v3/projects/f29295d0fb6141208e3a094c3be29944 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:19.133494 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4512610-0ef3-449b-aebd-f8adf0a1c33e None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:19.134201 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cd963-9774-4f97-aa27-4b35fad06f24 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:19.134870 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cd963-9774-4f97-aa27-4b35fad06f24 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:19.135165 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cd963-9774-4f97-aa27-4b35fad06f24 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:19.136819 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 82/166] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:19 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:19.137419 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cd963-9774-4f97-aa27-4b35fad06f24 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:19.203448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:19.251269 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:19.297575 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4512610-0ef3-449b-aebd-f8adf0a1c33e None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:19.336234 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5fc5ad-8d2c-4363-924a-fe76916a2c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:19.337066 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fc5ad-8d2c-4363-924a-fe76916a2c3b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:19.337346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fc5ad-8d2c-4363-924a-fe76916a2c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:19.337620 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5fc5ad-8d2c-4363-924a-fe76916a2c3b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:19.353600 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5fc5ad-8d2c-4363-924a-fe76916a2c3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:19.365862 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-3f5fc5ad-8d2c-4363-924a-fe76916a2c3b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:53:19.365862 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:19.365862 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:19.365862 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:19.365862 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:19.365862 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:19.365862 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:19.365862 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:19.365862 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:19.365862 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:19.365862 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:19.365862 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:19.365862 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:19.365862 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:19.365862 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:19.365862 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:19.367491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:19.367491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:19.367491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:19.367491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:19.367491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:19.367491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:19.367491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:19.367491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:19.367491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:19.367491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:19.367491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:19.367491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:19.367491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:19.367491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:19.367491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:19.367491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:19.368491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:19.368491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:19.368491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:19.368491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:19.368491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:19.368491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:19.368491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:19.368491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:19.368491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:19.368491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:19.368491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:53:19.368491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:19.368491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 85/167] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 23 16:53:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:19.369164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cd963-9774-4f97-aa27-4b35fad06f24 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:19.371954 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37527ade-911e-4506-87d1-dbb4f6a7c794 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:19.372246 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37527ade-911e-4506-87d1-dbb4f6a7c794 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:19.372529 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37527ade-911e-4506-87d1-dbb4f6a7c794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:19.408210 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-37527ade-911e-4506-87d1-dbb4f6a7c794 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:19.409520 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-37527ade-911e-4506-87d1-dbb4f6a7c794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:19.432813 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd96e4-51c1-4885-8371-5f5f05a13f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:19.433494 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd96e4-51c1-4885-8371-5f5f05a13f24 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:19.433775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd96e4-51c1-4885-8371-5f5f05a13f24 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:19.434036 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0abd96e4-51c1-4885-8371-5f5f05a13f24 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:19.434873 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd96e4-51c1-4885-8371-5f5f05a13f24 None admin] RBAC: Adding query filter params (role.id=72cc26a1a66c4fc183b39fad4cb38431, scope.project.id=f29295d0fb6141208e3a094c3be29944, user.id=99a4070a60cc4ae8a1981ac1c10668fe) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:19.435209 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd96e4-51c1-4885-8371-5f5f05a13f24 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:19.436703 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0abd96e4-51c1-4885-8371-5f5f05a13f24 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:19.449651 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 83/168] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:53:19 2022] GET /identity/v3/role_assignments?user.id=99a4070a60cc4ae8a1981ac1c10668fe&scope.project.id=f29295d0fb6141208e3a094c3be29944&role.id=72cc26a1a66c4fc183b39fad4cb38431 => generated 608 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:19.450353 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd96e4-51c1-4885-8371-5f5f05a13f24 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:19.554996 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 86/169] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:19.615165 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37527ade-911e-4506-87d1-dbb4f6a7c794 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:19.650161 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0abd96e4-51c1-4885-8371-5f5f05a13f24 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:19.667360 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:19.703711 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf444f1-01a8-4913-a781-0f5b72d1a4dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:19.704409 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf444f1-01a8-4913-a781-0f5b72d1a4dd None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:19.704719 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf444f1-01a8-4913-a781-0f5b72d1a4dd None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:19.704993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf444f1-01a8-4913-a781-0f5b72d1a4dd None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:19.705754 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf444f1-01a8-4913-a781-0f5b72d1a4dd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:19.706059 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf444f1-01a8-4913-a781-0f5b72d1a4dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:19.707510 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf444f1-01a8-4913-a781-0f5b72d1a4dd None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:19.717199 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 84/170] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:53:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:19.727557 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf444f1-01a8-4913-a781-0f5b72d1a4dd None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:19.734466 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:19.857744 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37527ade-911e-4506-87d1-dbb4f6a7c794 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:19.866623 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf444f1-01a8-4913-a781-0f5b72d1a4dd None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:19.914969 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f082bb-8bb7-490c-a2a5-de21134d5239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:19.915905 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f082bb-8bb7-490c-a2a5-de21134d5239 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:19.916185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f082bb-8bb7-490c-a2a5-de21134d5239 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:19.916457 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f082bb-8bb7-490c-a2a5-de21134d5239 None admin] PATH_INFO: `/v3/users/99a4070a60cc4ae8a1981ac1c10668fe` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:19.917671 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f082bb-8bb7-490c-a2a5-de21134d5239 None admin] RBAC: Authorizing `identity:get_user(user_id=99a4070a60cc4ae8a1981ac1c10668fe)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:19.919157 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f082bb-8bb7-490c-a2a5-de21134d5239 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:19.920959 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 85/171] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:19 2022] GET /identity/v3/users/99a4070a60cc4ae8a1981ac1c10668fe => generated 275 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:19.926466 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f082bb-8bb7-490c-a2a5-de21134d5239 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:19.945493 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a429c-c55f-49f4-a46b-8280df4e7d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:19.946317 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a429c-c55f-49f4-a46b-8280df4e7d78 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:19.946599 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a429c-c55f-49f4-a46b-8280df4e7d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:19.946869 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1a429c-c55f-49f4-a46b-8280df4e7d78 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:19.963318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:19.973049 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1a429c-c55f-49f4-a46b-8280df4e7d78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:19.982007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-5b1a429c-c55f-49f4-a46b-8280df4e7d78 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:19.982007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:19.982007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:19.982007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:19.982007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:19.982007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:19.982007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:19.982007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:19.982007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:19.982007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:19.982007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:19.982007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:19.982007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:19.982007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:19.982007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:19.982007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:19.982941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:19.982941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:19.982941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:19.982941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:19.982941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:19.982941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:19.982941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:19.982941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:19.982941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:19.982941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:19.982941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:19.982941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:19.982941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:19.982941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:19.982941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:19.982941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:19.983736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:19.983736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:19.983736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:19.983736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:19.983736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:19.983736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:19.983736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:19.983736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:19.983736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:19.983736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:19.983736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:19.983736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:19.987586 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 87/172] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 373 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:19.994342 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a429c-c55f-49f4-a46b-8280df4e7d78 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:20.059627 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f082bb-8bb7-490c-a2a5-de21134d5239 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:20.116548 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf76dde-c07c-4d5e-b6a1-e4f21d8c61d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:20.117145 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf76dde-c07c-4d5e-b6a1-e4f21d8c61d0 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:20.117425 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf76dde-c07c-4d5e-b6a1-e4f21d8c61d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:20.117699 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf76dde-c07c-4d5e-b6a1-e4f21d8c61d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:20.159023 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1a429c-c55f-49f4-a46b-8280df4e7d78 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:20.173465 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:20.216475 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8ac21-b787-43a7-8b3e-64cefc5af1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:20.217169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8ac21-b787-43a7-8b3e-64cefc5af1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:20.217455 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8ac21-b787-43a7-8b3e-64cefc5af1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:20.217733 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc8ac21-b787-43a7-8b3e-64cefc5af1a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:20.218498 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc8ac21-b787-43a7-8b3e-64cefc5af1a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:20.218808 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc8ac21-b787-43a7-8b3e-64cefc5af1a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:20.220253 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc8ac21-b787-43a7-8b3e-64cefc5af1a8 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:20.221639 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf76dde-c07c-4d5e-b6a1-e4f21d8c61d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:20.223349 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf76dde-c07c-4d5e-b6a1-e4f21d8c61d0 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:20.235284 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 88/173] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:20.240569 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8ac21-b787-43a7-8b3e-64cefc5af1a8 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:20.266272 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:20.375417 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc8ac21-b787-43a7-8b3e-64cefc5af1a8 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:20.386306 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 86/174] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 460 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 16:53:20.387504 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86527587-fc1e-4c3e-a795-c44b999edf21 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:20.387790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86527587-fc1e-4c3e-a795-c44b999edf21 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:20.388063 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86527587-fc1e-4c3e-a795-c44b999edf21 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:20.389566 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 87/175] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:20.394357 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86527587-fc1e-4c3e-a795-c44b999edf21 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:20.411607 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3580b916-0d48-49d5-954c-a8dea6db1fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:20.412619 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580b916-0d48-49d5-954c-a8dea6db1fd2 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:20.412903 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580b916-0d48-49d5-954c-a8dea6db1fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:20.413177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3580b916-0d48-49d5-954c-a8dea6db1fd2 None admin] PATH_INFO: `/v3/users/99a4070a60cc4ae8a1981ac1c10668fe` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:20.414401 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3580b916-0d48-49d5-954c-a8dea6db1fd2 None admin] RBAC: Authorizing `identity:get_user(user_id=99a4070a60cc4ae8a1981ac1c10668fe)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:20.417961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3580b916-0d48-49d5-954c-a8dea6db1fd2 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:20.419653 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 89/176] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:20 2022] GET /identity/v3/users/99a4070a60cc4ae8a1981ac1c10668fe => generated 275 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:20.424456 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbce9a-8b97-4cda-b831-6253a3f45b4b None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:20.424744 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbce9a-8b97-4cda-b831-6253a3f45b4b None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:20.425020 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fbce9a-8b97-4cda-b831-6253a3f45b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:20.459177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-e7fbce9a-8b97-4cda-b831-6253a3f45b4b None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:20.460456 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-e7fbce9a-8b97-4cda-b831-6253a3f45b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:20.493960 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:20.537015 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 90/177] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:20.539229 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbce9a-8b97-4cda-b831-6253a3f45b4b None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:20.627612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86527587-fc1e-4c3e-a795-c44b999edf21 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:20.660682 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:20.672961 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70e129-d38b-402a-9db2-7c416a46809f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:20.673841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70e129-d38b-402a-9db2-7c416a46809f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:20.674123 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70e129-d38b-402a-9db2-7c416a46809f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:20.674398 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c70e129-d38b-402a-9db2-7c416a46809f None admin] PATH_INFO: `/v3/projects/f29295d0fb6141208e3a094c3be29944` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:20.675824 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c70e129-d38b-402a-9db2-7c416a46809f None admin] RBAC: Authorizing `identity:get_project(project_id=f29295d0fb6141208e3a094c3be29944)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:20.677391 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c70e129-d38b-402a-9db2-7c416a46809f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:20.679176 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 88/178] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:20 2022] GET /identity/v3/projects/f29295d0fb6141208e3a094c3be29944 => generated 298 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:20.679912 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fbce9a-8b97-4cda-b831-6253a3f45b4b None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:20.680032 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70e129-d38b-402a-9db2-7c416a46809f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:20.715922 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a432fc46-9105-4d85-b9a0-96e820379902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:20.716885 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432fc46-9105-4d85-b9a0-96e820379902 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:20.717342 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432fc46-9105-4d85-b9a0-96e820379902 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:20.717785 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a432fc46-9105-4d85-b9a0-96e820379902 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:20.757237 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:20.790186 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c70e129-d38b-402a-9db2-7c416a46809f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:20.794484 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a432fc46-9105-4d85-b9a0-96e820379902 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:20.796126 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a432fc46-9105-4d85-b9a0-96e820379902 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:20.827989 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8bbf2c-b7ae-4904-9e59-f94b2f826edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:20.828792 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bbf2c-b7ae-4904-9e59-f94b2f826edf None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:20.829073 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bbf2c-b7ae-4904-9e59-f94b2f826edf None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:20.829344 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8bbf2c-b7ae-4904-9e59-f94b2f826edf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:20.844228 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8bbf2c-b7ae-4904-9e59-f94b2f826edf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:20.855175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-0b8bbf2c-b7ae-4904-9e59-f94b2f826edf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:53:20.855175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:20.855175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:20.855175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:20.855175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:20.855175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:20.855175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:20.855175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:20.855175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:20.855175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:20.855175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:20.855175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:20.855175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:20.855175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:20.855175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:20.855175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:20.856060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:20.856060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:20.856060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:20.856060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:20.856060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:20.856060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:20.856060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:20.856060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:20.856060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:20.856060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:20.856060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:20.856060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:20.856060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:20.856060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:20.856060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:20.856060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:20.856827 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:20.856827 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:20.856827 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:20.856827 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:20.856827 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:20.856827 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:20.856827 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:20.856827 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:20.856827 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:20.856827 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:20.856827 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:53:20.856827 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:20.859020 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 89/179] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:53:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:20.864092 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea35e2-599f-428d-a0d0-f3b21d36023c None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:20.864092 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea35e2-599f-428d-a0d0-f3b21d36023c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:20.864262 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbea35e2-599f-428d-a0d0-f3b21d36023c None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:20.865755 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 90/180] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:20.867960 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea35e2-599f-428d-a0d0-f3b21d36023c None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:20.911390 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 91/181] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:53:20.912264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a432fc46-9105-4d85-b9a0-96e820379902 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:20.937106 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:20.980857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:20.993820 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbea35e2-599f-428d-a0d0-f3b21d36023c None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:21.017496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a432fc46-9105-4d85-b9a0-96e820379902 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:21.028939 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef851c4-576c-4e0d-8a9a-ad66cae64c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:21.029599 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef851c4-576c-4e0d-8a9a-ad66cae64c18 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:21.029875 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef851c4-576c-4e0d-8a9a-ad66cae64c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:21.030141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef851c4-576c-4e0d-8a9a-ad66cae64c18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:21.031116 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef851c4-576c-4e0d-8a9a-ad66cae64c18 None admin] RBAC: Adding query filter params (role.id=6c30aca17fad4c4fa40b9461c5bda291, scope.project.id=f29295d0fb6141208e3a094c3be29944, user.id=99a4070a60cc4ae8a1981ac1c10668fe) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:21.031656 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef851c4-576c-4e0d-8a9a-ad66cae64c18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:21.032993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef851c4-576c-4e0d-8a9a-ad66cae64c18 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:21.039569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 91/182] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:53:20 2022] GET /identity/v3/role_assignments?user.id=99a4070a60cc4ae8a1981ac1c10668fe&scope.project.id=f29295d0fb6141208e3a094c3be29944&role.id=6c30aca17fad4c4fa40b9461c5bda291 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:21.040627 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287eda7-577d-431e-8681-3e22c4ebdba7 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:21.040907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287eda7-577d-431e-8681-3e22c4ebdba7 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:21.041175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b287eda7-577d-431e-8681-3e22c4ebdba7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:21.059248 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-b287eda7-577d-431e-8681-3e22c4ebdba7 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:21.060624 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-b287eda7-577d-431e-8681-3e22c4ebdba7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:21.066892 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267666b0-99e1-4a55-8ab7-109c39431e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:21.067996 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267666b0-99e1-4a55-8ab7-109c39431e23 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:21.068582 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267666b0-99e1-4a55-8ab7-109c39431e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:21.069134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267666b0-99e1-4a55-8ab7-109c39431e23 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:21.070899 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267666b0-99e1-4a55-8ab7-109c39431e23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:21.071470 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267666b0-99e1-4a55-8ab7-109c39431e23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:21.073739 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267666b0-99e1-4a55-8ab7-109c39431e23 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:21.086839 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 92/183] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:21.088076 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267666b0-99e1-4a55-8ab7-109c39431e23 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:21.150850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 92/184] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:21.155717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea76187-e8d3-4d4f-8c08-8837b3602a3d None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:21.155717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea76187-e8d3-4d4f-8c08-8837b3602a3d None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:21.155717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea76187-e8d3-4d4f-8c08-8837b3602a3d None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:21.155717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 93/185] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:21.155717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea76187-e8d3-4d4f-8c08-8837b3602a3d None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:21.221766 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:21.233969 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267666b0-99e1-4a55-8ab7-109c39431e23 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:21.266673 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:21.274544 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103ebe49-76c8-434e-ac0b-deac88bb2494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:21.275769 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ebe49-76c8-434e-ac0b-deac88bb2494 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:21.276256 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ebe49-76c8-434e-ac0b-deac88bb2494 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:21.276624 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103ebe49-76c8-434e-ac0b-deac88bb2494 None admin] PATH_INFO: `/v3/projects/15bd2384217841e5bc06a9047dbfb59d` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:21.277845 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ebe49-76c8-434e-ac0b-deac88bb2494 None admin] RBAC: Authorizing `identity:get_project(project_id=15bd2384217841e5bc06a9047dbfb59d)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:21.279337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103ebe49-76c8-434e-ac0b-deac88bb2494 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:21.281040 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 93/186] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:21 2022] GET /identity/v3/projects/15bd2384217841e5bc06a9047dbfb59d => generated 312 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:21.282204 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0eab2a-27f2-45f3-8c33-bf2d6e9cca33 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:21.282481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0eab2a-27f2-45f3-8c33-bf2d6e9cca33 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:21.282749 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0eab2a-27f2-45f3-8c33-bf2d6e9cca33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:21.289303 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea76187-e8d3-4d4f-8c08-8837b3602a3d None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:21.308374 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-8d0eab2a-27f2-45f3-8c33-bf2d6e9cca33 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:21.309591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-8d0eab2a-27f2-45f3-8c33-bf2d6e9cca33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:21.337465 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd2fbe-ff47-4e6c-bd63-6e3732ca9229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:21.338590 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd2fbe-ff47-4e6c-bd63-6e3732ca9229 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:21.338867 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd2fbe-ff47-4e6c-bd63-6e3732ca9229 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:21.339163 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bd2fbe-ff47-4e6c-bd63-6e3732ca9229 None admin] PATH_INFO: `/v3/users/c5ba91cf43124aa98c81468af3f0f4fc` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:21.355140 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd2fbe-ff47-4e6c-bd63-6e3732ca9229 None admin] RBAC: Authorizing `identity:get_user(user_id=c5ba91cf43124aa98c81468af3f0f4fc)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:21.356661 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bd2fbe-ff47-4e6c-bd63-6e3732ca9229 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:21.358582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 94/187] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:21 2022] GET /identity/v3/users/c5ba91cf43124aa98c81468af3f0f4fc => generated 283 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:21.359562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd2fbe-ff47-4e6c-bd63-6e3732ca9229 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:21.386929 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 94/188] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:21.388111 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0eab2a-27f2-45f3-8c33-bf2d6e9cca33 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:21.473876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bd2fbe-ff47-4e6c-bd63-6e3732ca9229 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:21.477323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:21.508605 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0eab2a-27f2-45f3-8c33-bf2d6e9cca33 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:21.509953 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6d5c7-64ca-47af-80a2-a7c14b108fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:21.510748 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6d5c7-64ca-47af-80a2-a7c14b108fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:21.511061 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6d5c7-64ca-47af-80a2-a7c14b108fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:21.511334 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a6d5c7-64ca-47af-80a2-a7c14b108fd6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:21.512500 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a6d5c7-64ca-47af-80a2-a7c14b108fd6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:21.513953 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9a6d5c7-64ca-47af-80a2-a7c14b108fd6 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:21.515589 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 95/189] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:53:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:21.516347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6d5c7-64ca-47af-80a2-a7c14b108fd6 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:21.542367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:21.554680 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61667591-c4d2-47ee-a90e-424a0b9290e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:21.555414 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61667591-c4d2-47ee-a90e-424a0b9290e5 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:21.555695 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61667591-c4d2-47ee-a90e-424a0b9290e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:21.555967 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61667591-c4d2-47ee-a90e-424a0b9290e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:21.556818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61667591-c4d2-47ee-a90e-424a0b9290e5 None admin] RBAC: Adding query filter params (role.id=72cc26a1a66c4fc183b39fad4cb38431, scope.project.id=15bd2384217841e5bc06a9047dbfb59d, user.id=99a4070a60cc4ae8a1981ac1c10668fe) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:21.557126 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61667591-c4d2-47ee-a90e-424a0b9290e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:21.558624 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61667591-c4d2-47ee-a90e-424a0b9290e5 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:21.566687 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 95/190] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:53:21 2022] GET /identity/v3/role_assignments?user.id=99a4070a60cc4ae8a1981ac1c10668fe&scope.project.id=15bd2384217841e5bc06a9047dbfb59d&role.id=72cc26a1a66c4fc183b39fad4cb38431 => generated 608 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:21.567412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61667591-c4d2-47ee-a90e-424a0b9290e5 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:21.644651 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a6d5c7-64ca-47af-80a2-a7c14b108fd6 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:21.684260 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61667591-c4d2-47ee-a90e-424a0b9290e5 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:21.684451 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f176a7-e67a-4b91-961a-181bcd2cad30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:21.685121 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f176a7-e67a-4b91-961a-181bcd2cad30 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:21.685401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f176a7-e67a-4b91-961a-181bcd2cad30 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:21.685669 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f176a7-e67a-4b91-961a-181bcd2cad30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:21.725264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21895f-3683-498f-843e-cafbdda28881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:21.726459 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21895f-3683-498f-843e-cafbdda28881 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:21.727068 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21895f-3683-498f-843e-cafbdda28881 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:21.727635 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa21895f-3683-498f-843e-cafbdda28881 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:21.737662 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:21.742075 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa21895f-3683-498f-843e-cafbdda28881 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:21.751043 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-aa21895f-3683-498f-843e-cafbdda28881 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:53:21.751043 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:21.751043 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:21.751043 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:21.751043 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:21.751043 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:21.751043 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:21.751043 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:21.751043 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:21.751043 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:21.751043 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:21.751043 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:21.751043 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:21.751043 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:21.751043 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:21.751043 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:21.752029 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:21.752029 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:21.752029 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:21.752029 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:21.752029 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:21.752029 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:21.752029 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:21.752029 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:21.752029 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:21.752029 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:21.752029 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:21.752029 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:21.752029 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:21.752029 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:21.752029 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:21.752029 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:21.753279 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:21.753279 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:21.753279 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:21.753279 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:21.753279 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:21.753279 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:21.753279 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:21.753279 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:21.753279 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:21.753279 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:21.753279 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:53:21.753279 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:21.753279 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 96/191] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 23 16:53:21 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:21.753279 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21895f-3683-498f-843e-cafbdda28881 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:21.773834 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f176a7-e67a-4b91-961a-181bcd2cad30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:21.775452 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f176a7-e67a-4b91-961a-181bcd2cad30 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:21.819341 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:21.878628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa21895f-3683-498f-843e-cafbdda28881 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:21.890660 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 96/192] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:53:21.892052 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564e70d-fc13-45ac-8b50-3deccb9225db None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:21.892339 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564e70d-fc13-45ac-8b50-3deccb9225db None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:21.892612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5564e70d-fc13-45ac-8b50-3deccb9225db None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:21.895096 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 97/193] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:21.895096 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564e70d-fc13-45ac-8b50-3deccb9225db None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:21.913017 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff44fce-62bb-48c5-a4a7-c9b768fbe27e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:21.913685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff44fce-62bb-48c5-a4a7-c9b768fbe27e None admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:21.913971 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff44fce-62bb-48c5-a4a7-c9b768fbe27e None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:21.914241 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff44fce-62bb-48c5-a4a7-c9b768fbe27e None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:21.914974 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff44fce-62bb-48c5-a4a7-c9b768fbe27e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:21.916421 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff44fce-62bb-48c5-a4a7-c9b768fbe27e None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:21.938005 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 97/194] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 23 16:53:21 2022] POST /identity/v3/users => generated 297 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:53:21.938823 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff44fce-62bb-48c5-a4a7-c9b768fbe27e None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:21.979215 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:22.018394 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5564e70d-fc13-45ac-8b50-3deccb9225db None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:22.058943 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dff22c-1a19-4b68-8d6c-d4e54a57ada4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:22.059632 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dff22c-1a19-4b68-8d6c-d4e54a57ada4 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:22.059918 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dff22c-1a19-4b68-8d6c-d4e54a57ada4 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:22.060189 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dff22c-1a19-4b68-8d6c-d4e54a57ada4 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:22.060380 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff44fce-62bb-48c5-a4a7-c9b768fbe27e None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:22.060951 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dff22c-1a19-4b68-8d6c-d4e54a57ada4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:22.061261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dff22c-1a19-4b68-8d6c-d4e54a57ada4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:22.062707 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dff22c-1a19-4b68-8d6c-d4e54a57ada4 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:22.070772 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:22.072039 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 98/195] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:53:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:22.073565 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8babc-92c2-45c4-8608-140603cb7900 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:22.073864 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8babc-92c2-45c4-8608-140603cb7900 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:22.074141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea8babc-92c2-45c4-8608-140603cb7900 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:22.092690 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-dea8babc-92c2-45c4-8608-140603cb7900 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:22.093917 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-dea8babc-92c2-45c4-8608-140603cb7900 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:22.103258 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3628af4-624d-4266-af2a-a80664cbe576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:22.104180 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3628af4-624d-4266-af2a-a80664cbe576 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:22.104462 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3628af4-624d-4266-af2a-a80664cbe576 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:22.105122 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3628af4-624d-4266-af2a-a80664cbe576 None admin] PATH_INFO: `/v3/projects/4206726f0af54455a4cb03012949ffb5` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:22.106747 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3628af4-624d-4266-af2a-a80664cbe576 None admin] RBAC: Authorizing `identity:get_project(project_id=4206726f0af54455a4cb03012949ffb5)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:22.108455 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3628af4-624d-4266-af2a-a80664cbe576 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:22.110440 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 98/196] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:21 2022] GET /identity/v3/projects/4206726f0af54455a4cb03012949ffb5 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:22.112096 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3628af4-624d-4266-af2a-a80664cbe576 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:22.174158 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 99/197] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:22.175853 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8babc-92c2-45c4-8608-140603cb7900 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:22.242032 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3628af4-624d-4266-af2a-a80664cbe576 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:22.255069 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:22.286017 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174258b-6cbe-4cf5-be1e-cf1cf553fa45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:22.287061 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174258b-6cbe-4cf5-be1e-cf1cf553fa45 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:22.287585 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174258b-6cbe-4cf5-be1e-cf1cf553fa45 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:22.288108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6174258b-6cbe-4cf5-be1e-cf1cf553fa45 None admin] PATH_INFO: `/v3/users/c5ba91cf43124aa98c81468af3f0f4fc` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:22.289650 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174258b-6cbe-4cf5-be1e-cf1cf553fa45 None admin] RBAC: Authorizing `identity:get_user(user_id=c5ba91cf43124aa98c81468af3f0f4fc)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:22.291227 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6174258b-6cbe-4cf5-be1e-cf1cf553fa45 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:22.292981 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 99/198] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:22 2022] GET /identity/v3/users/c5ba91cf43124aa98c81468af3f0f4fc => generated 283 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:22.293861 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174258b-6cbe-4cf5-be1e-cf1cf553fa45 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:22.304388 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea8babc-92c2-45c4-8608-140603cb7900 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:22.346203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db7b95-4638-4d7c-b40c-5b8868f673a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:22.346876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db7b95-4638-4d7c-b40c-5b8868f673a4 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:22.347189 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db7b95-4638-4d7c-b40c-5b8868f673a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:22.347471 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75db7b95-4638-4d7c-b40c-5b8868f673a4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:22.348303 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75db7b95-4638-4d7c-b40c-5b8868f673a4 None admin] RBAC: Adding query filter params (role.id=17bbe7cc1c3641509de5e9dca8534102, scope.project.id=4206726f0af54455a4cb03012949ffb5, user.id=c5ba91cf43124aa98c81468af3f0f4fc) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:22.348615 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75db7b95-4638-4d7c-b40c-5b8868f673a4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:22.350395 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75db7b95-4638-4d7c-b40c-5b8868f673a4 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:22.350817 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:22.357293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 100/199] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:53:22 2022] GET /identity/v3/role_assignments?user.id=c5ba91cf43124aa98c81468af3f0f4fc&scope.project.id=4206726f0af54455a4cb03012949ffb5&role.id=17bbe7cc1c3641509de5e9dca8534102 => generated 259 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:22.358127 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db7b95-4638-4d7c-b40c-5b8868f673a4 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:22.416090 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6174258b-6cbe-4cf5-be1e-cf1cf553fa45 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:22.460745 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95eb0e-c21c-4cf8-a59c-d1214b8e3d12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:22.461578 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95eb0e-c21c-4cf8-a59c-d1214b8e3d12 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:22.461847 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95eb0e-c21c-4cf8-a59c-d1214b8e3d12 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:22.462113 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b95eb0e-c21c-4cf8-a59c-d1214b8e3d12 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:22.492663 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b95eb0e-c21c-4cf8-a59c-d1214b8e3d12 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:22.501184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-4b95eb0e-c21c-4cf8-a59c-d1214b8e3d12 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 16:53:22.501184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:22.501184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:22.501184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:22.501184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:22.501184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:22.501184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:22.501184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:22.501184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:22.501184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:22.501184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:22.501184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:22.501184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:22.501184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:22.501184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:22.501184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:22.502144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:22.502144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:22.502144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:22.502144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:22.502144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:22.502144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:22.502144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:22.502144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:22.502144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:22.502144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:22.502144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:22.502144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:22.502144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:22.502144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:22.502144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:22.502144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:22.502953 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:22.502953 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:22.502953 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:22.502953 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:22.502953 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:22.502953 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:22.502953 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:22.502953 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:22.502953 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:22.502953 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:22.502953 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 16:53:22.502953 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:22.502953 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 100/200] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:22 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:22.509915 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95eb0e-c21c-4cf8-a59c-d1214b8e3d12 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:22.608422 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75db7b95-4638-4d7c-b40c-5b8868f673a4 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:22.644168 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:22.656243 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec766cc1-924e-4c88-b2a6-f612470255dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:22.663161 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec766cc1-924e-4c88-b2a6-f612470255dd None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:22.663512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec766cc1-924e-4c88-b2a6-f612470255dd None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:22.663868 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec766cc1-924e-4c88-b2a6-f612470255dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:22.668892 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b95eb0e-c21c-4cf8-a59c-d1214b8e3d12 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:22.679305 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:22.731545 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d6e2f-eed5-4539-a000-15413f3e3819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:22.732224 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d6e2f-eed5-4539-a000-15413f3e3819 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:22.732507 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d6e2f-eed5-4539-a000-15413f3e3819 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:22.732776 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1d6e2f-eed5-4539-a000-15413f3e3819 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:22.733525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d6e2f-eed5-4539-a000-15413f3e3819 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:22.733826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d6e2f-eed5-4539-a000-15413f3e3819 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:22.735250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1d6e2f-eed5-4539-a000-15413f3e3819 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:22.755007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 101/201] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:22 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:22.763870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d6e2f-eed5-4539-a000-15413f3e3819 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:22.802518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec766cc1-924e-4c88-b2a6-f612470255dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:22.804715 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec766cc1-924e-4c88-b2a6-f612470255dd None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:22.881399 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1d6e2f-eed5-4539-a000-15413f3e3819 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:22.920300 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf002b-6687-4593-99ed-e03d30997b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:22.921532 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf002b-6687-4593-99ed-e03d30997b8c None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:22.922008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf002b-6687-4593-99ed-e03d30997b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:22.922484 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bf002b-6687-4593-99ed-e03d30997b8c None admin] PATH_INFO: `/v3/users/0de58fd59284495d9eab8ab28e53612c` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:22.924464 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf002b-6687-4593-99ed-e03d30997b8c None admin] RBAC: Authorizing `identity:get_user(user_id=0de58fd59284495d9eab8ab28e53612c)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:22.926261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3bf002b-6687-4593-99ed-e03d30997b8c None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:22.928515 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 102/202] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:22 2022] GET /identity/v3/users/0de58fd59284495d9eab8ab28e53612c => generated 289 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:22.934392 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf002b-6687-4593-99ed-e03d30997b8c None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:22.973161 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:23.045142 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bf002b-6687-4593-99ed-e03d30997b8c None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:23.063398 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 101/203] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 705 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:53:23.073827 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec766cc1-924e-4c88-b2a6-f612470255dd None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:23.087118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209090ec-358e-458f-8e5d-94a261d3f138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:23.088072 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209090ec-358e-458f-8e5d-94a261d3f138 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:23.088640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209090ec-358e-458f-8e5d-94a261d3f138 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:23.089099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209090ec-358e-458f-8e5d-94a261d3f138 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:23.158463 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:23.160705 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209090ec-358e-458f-8e5d-94a261d3f138 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:23.162871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209090ec-358e-458f-8e5d-94a261d3f138 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:23.199258 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:23.270342 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 103/204] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 16:53:23.284616 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209090ec-358e-458f-8e5d-94a261d3f138 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:23.362371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec766cc1-924e-4c88-b2a6-f612470255dd None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:23.411963 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209090ec-358e-458f-8e5d-94a261d3f138 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:23.457335 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f150625-322b-4b71-9e49-d20de4961578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:23.458637 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f150625-322b-4b71-9e49-d20de4961578 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:23.459208 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f150625-322b-4b71-9e49-d20de4961578 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:23.459668 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f150625-322b-4b71-9e49-d20de4961578 None admin] PATH_INFO: `/v3/projects/f29295d0fb6141208e3a094c3be29944` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:23.461328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f150625-322b-4b71-9e49-d20de4961578 None admin] RBAC: Authorizing `identity:get_project(project_id=f29295d0fb6141208e3a094c3be29944)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:23.463435 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f150625-322b-4b71-9e49-d20de4961578 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:23.465355 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 104/205] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:23 2022] GET /identity/v3/projects/f29295d0fb6141208e3a094c3be29944 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:23.478195 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ccacb-c962-479c-bf6e-02d5a50ce95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:23.478195 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ccacb-c962-479c-bf6e-02d5a50ce95f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:23.478195 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ccacb-c962-479c-bf6e-02d5a50ce95f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:23.478195 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737ccacb-c962-479c-bf6e-02d5a50ce95f None admin] PATH_INFO: `/v3/projects/4206726f0af54455a4cb03012949ffb5` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:23.480571 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ccacb-c962-479c-bf6e-02d5a50ce95f None admin] RBAC: Authorizing `identity:get_project(project_id=4206726f0af54455a4cb03012949ffb5)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:23.480993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f150625-322b-4b71-9e49-d20de4961578 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:23.482079 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737ccacb-c962-479c-bf6e-02d5a50ce95f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:23.483845 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 102/206] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:23 2022] GET /identity/v3/projects/4206726f0af54455a4cb03012949ffb5 => generated 302 bytes in 410 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:23.484454 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:23.489790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ccacb-c962-479c-bf6e-02d5a50ce95f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:23.659260 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:23.668912 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737ccacb-c962-479c-bf6e-02d5a50ce95f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:23.682304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f150625-322b-4b71-9e49-d20de4961578 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:23.728379 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50507029-2261-4a8b-8cab-0ebfc1c73154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:23.729346 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50507029-2261-4a8b-8cab-0ebfc1c73154 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:23.729767 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50507029-2261-4a8b-8cab-0ebfc1c73154 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:23.730175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50507029-2261-4a8b-8cab-0ebfc1c73154 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:23.731176 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50507029-2261-4a8b-8cab-0ebfc1c73154 None admin] RBAC: Adding query filter params (role.id=6c30aca17fad4c4fa40b9461c5bda291, scope.project.id=4206726f0af54455a4cb03012949ffb5, user.id=c5ba91cf43124aa98c81468af3f0f4fc) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:23.731629 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50507029-2261-4a8b-8cab-0ebfc1c73154 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:23.734309 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50507029-2261-4a8b-8cab-0ebfc1c73154 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:23.745536 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 103/207] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:53:23 2022] GET /identity/v3/role_assignments?user.id=c5ba91cf43124aa98c81468af3f0f4fc&scope.project.id=4206726f0af54455a4cb03012949ffb5&role.id=6c30aca17fad4c4fa40b9461c5bda291 => generated 259 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:23.757863 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764262f9-81ff-47ec-ac57-9facab9a1b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:23.758816 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764262f9-81ff-47ec-ac57-9facab9a1b5a None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:23.759566 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764262f9-81ff-47ec-ac57-9facab9a1b5a None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:23.760006 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764262f9-81ff-47ec-ac57-9facab9a1b5a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:23.760987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764262f9-81ff-47ec-ac57-9facab9a1b5a None admin] RBAC: Adding query filter params (role.id=9d4694dd70b94d68acc78c53ca498bbf, scope.project.id=f29295d0fb6141208e3a094c3be29944, user.id=0de58fd59284495d9eab8ab28e53612c) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:23.761440 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764262f9-81ff-47ec-ac57-9facab9a1b5a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:23.763049 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-764262f9-81ff-47ec-ac57-9facab9a1b5a None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:23.769981 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 105/208] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:53:23 2022] GET /identity/v3/role_assignments?user.id=0de58fd59284495d9eab8ab28e53612c&scope.project.id=f29295d0fb6141208e3a094c3be29944&role.id=9d4694dd70b94d68acc78c53ca498bbf => generated 608 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:24.516389 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe09de-9250-4ece-a004-162aa42cb58c None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:24.516897 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe09de-9250-4ece-a004-162aa42cb58c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:24.516969 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9fe09de-9250-4ece-a004-162aa42cb58c None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:24.518499 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 104/209] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:24.524480 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f92fb0-6cff-4f58-a230-20f538fff66b None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:24.524915 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f92fb0-6cff-4f58-a230-20f538fff66b None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:24.525091 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f92fb0-6cff-4f58-a230-20f538fff66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:24.542808 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-33f92fb0-6cff-4f58-a230-20f538fff66b None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:24.544060 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-33f92fb0-6cff-4f58-a230-20f538fff66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:24.606743 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:24.615099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 106/210] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:24.627946 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84e23e-53a2-4679-9e96-8230446a9d7a None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:24.628251 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84e23e-53a2-4679-9e96-8230446a9d7a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:24.628528 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84e23e-53a2-4679-9e96-8230446a9d7a None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:24.629902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 105/211] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:24.635611 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22035a4-06e1-439c-a21b-49f310e2095a None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:24.635897 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22035a4-06e1-439c-a21b-49f310e2095a None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:24.636174 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22035a4-06e1-439c-a21b-49f310e2095a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:24.644397 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84e23e-53a2-4679-9e96-8230446a9d7a None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:24.656708 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-c22035a4-06e1-439c-a21b-49f310e2095a None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:24.658830 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-c22035a4-06e1-439c-a21b-49f310e2095a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:24.750143 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 107/212] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:24.764546 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84e23e-53a2-4679-9e96-8230446a9d7a None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:24.777271 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22035a4-06e1-439c-a21b-49f310e2095a None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:24.806281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58282cb4-1fee-4248-a714-a3d50d08d501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:24.808038 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58282cb4-1fee-4248-a714-a3d50d08d501 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:24.808756 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58282cb4-1fee-4248-a714-a3d50d08d501 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:24.809202 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58282cb4-1fee-4248-a714-a3d50d08d501 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:24.823092 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58282cb4-1fee-4248-a714-a3d50d08d501 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:24.834433 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-58282cb4-1fee-4248-a714-a3d50d08d501 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:24.834433 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:24.834433 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:24.834433 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:24.834433 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:24.834433 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:24.834433 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:24.834433 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:24.834433 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:24.834433 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:24.834433 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:24.834433 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:24.834433 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:24.834433 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:24.834433 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:24.834433 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:24.835363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:24.835363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:24.835363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:24.835363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:24.835363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:24.835363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:24.835363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:24.835363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:24.835363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:24.835363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:24.835363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:24.835363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:24.835363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:24.835363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:24.835363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:24.835363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:24.836157 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:24.836157 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:24.836157 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:24.836157 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:24.836157 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:24.836157 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:24.836157 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:24.836157 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:24.836157 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:24.836157 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:24.836157 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:24.836157 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:24.839078 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 106/213] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:53:24.844792 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58282cb4-1fee-4248-a714-a3d50d08d501 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:24.864472 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:24.892584 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22035a4-06e1-439c-a21b-49f310e2095a None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:24.921214 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:24.931732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4bd9775-b86e-4f65-b95b-5edb167afbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:24.932542 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd9775-b86e-4f65-b95b-5edb167afbb1 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:24.932825 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd9775-b86e-4f65-b95b-5edb167afbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:24.933093 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4bd9775-b86e-4f65-b95b-5edb167afbb1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:24.934334 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd9775-b86e-4f65-b95b-5edb167afbb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:24.935776 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4bd9775-b86e-4f65-b95b-5edb167afbb1 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:24.937529 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 108/214] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:53:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:24.938539 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212cadb-7ac5-4f80-b2b9-233bf87bdc7c None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:24.938821 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212cadb-7ac5-4f80-b2b9-233bf87bdc7c None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:24.939118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6212cadb-7ac5-4f80-b2b9-233bf87bdc7c None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:24.940451 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 109/215] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:24 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:24.944317 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212cadb-7ac5-4f80-b2b9-233bf87bdc7c None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:24.957699 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58282cb4-1fee-4248-a714-a3d50d08d501 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:24.997121 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d70ffc-68fe-45a7-87d6-edbc6b09bddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:24.997806 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d70ffc-68fe-45a7-87d6-edbc6b09bddd None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:24.998090 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d70ffc-68fe-45a7-87d6-edbc6b09bddd None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:24.998367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d70ffc-68fe-45a7-87d6-edbc6b09bddd None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:24.999160 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d70ffc-68fe-45a7-87d6-edbc6b09bddd None admin] RBAC: Adding query filter params (name=member) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:24.999477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d70ffc-68fe-45a7-87d6-edbc6b09bddd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:25.000941 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d70ffc-68fe-45a7-87d6-edbc6b09bddd None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:25.011077 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 107/216] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:25.016790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969d236-32df-4dca-8e61-23cd413bbf3e None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:25.017079 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969d236-32df-4dca-8e61-23cd413bbf3e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:25.017358 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6969d236-32df-4dca-8e61-23cd413bbf3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:25.034162 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-6969d236-32df-4dca-8e61-23cd413bbf3e None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:25.035370 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-6969d236-32df-4dca-8e61-23cd413bbf3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:25.059680 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6212cadb-7ac5-4f80-b2b9-233bf87bdc7c None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:25.096837 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc5b507-46b3-4265-b557-ecf5c8bb4bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:25.097550 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5b507-46b3-4265-b557-ecf5c8bb4bda None admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:25.097832 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5b507-46b3-4265-b557-ecf5c8bb4bda None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:25.098108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc5b507-46b3-4265-b557-ecf5c8bb4bda None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:25.098815 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc5b507-46b3-4265-b557-ecf5c8bb4bda None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:25.100967 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc5b507-46b3-4265-b557-ecf5c8bb4bda None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:25.105332 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 108/217] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:25.106203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969d236-32df-4dca-8e61-23cd413bbf3e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:25.119445 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:25.139067 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 110/218] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 23 16:53:24 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:53:25.139989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc5b507-46b3-4265-b557-ecf5c8bb4bda None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:25.207984 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:25.245169 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6969d236-32df-4dca-8e61-23cd413bbf3e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:25.277045 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc5b507-46b3-4265-b557-ecf5c8bb4bda None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:25.291533 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fa4a9-6368-42f4-8879-bc905e421a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:25.291533 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fa4a9-6368-42f4-8879-bc905e421a68 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:25.291533 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fa4a9-6368-42f4-8879-bc905e421a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:25.291533 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47fa4a9-6368-42f4-8879-bc905e421a68 None admin] PATH_INFO: `/v3/users/19eff116df6a4c9a859e361fc6e9d908` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:25.308222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47fa4a9-6368-42f4-8879-bc905e421a68 None admin] RBAC: Authorizing `identity:get_user(user_id=19eff116df6a4c9a859e361fc6e9d908)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:25.309811 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b47fa4a9-6368-42f4-8879-bc905e421a68 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:25.311683 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 109/219] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:25 2022] GET /identity/v3/users/19eff116df6a4c9a859e361fc6e9d908 => generated 297 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:25.317569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fa4a9-6368-42f4-8879-bc905e421a68 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:25.332848 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69072c-c82d-4447-96f7-72b8c6c03ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:25.334181 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69072c-c82d-4447-96f7-72b8c6c03ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:25.334465 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69072c-c82d-4447-96f7-72b8c6c03ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:25.334989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc69072c-c82d-4447-96f7-72b8c6c03ff4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:25.354011 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc69072c-c82d-4447-96f7-72b8c6c03ff4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:25.363282 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-cc69072c-c82d-4447-96f7-72b8c6c03ff4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:53:25.363282 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:25.363282 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:25.363282 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:25.363282 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:25.363282 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:25.363282 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:25.363282 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:25.363282 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:25.363282 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:25.363282 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:25.363282 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:25.363282 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:25.363282 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:25.363282 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:25.363282 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:25.364225 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:25.364225 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:25.364225 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:25.364225 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:25.364225 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:25.364225 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:25.364225 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:25.364225 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:25.364225 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:25.364225 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:25.364225 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:25.364225 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:25.364225 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:25.364225 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:25.364225 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:25.364225 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:25.364993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:25.364993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:25.364993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:25.364993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:25.364993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:25.364993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:25.364993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:25.364993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:25.364993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:25.364993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:25.364993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:53:25.364993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:25.366511 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 111/220] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:53:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:25.371320 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69072c-c82d-4447-96f7-72b8c6c03ff4 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:25.396760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:25.441165 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47fa4a9-6368-42f4-8879-bc905e421a68 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:25.491385 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fd41d-6945-4a24-b09b-a80bd9f47494 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:25.494575 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fd41d-6945-4a24-b09b-a80bd9f47494 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:25.494575 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fd41d-6945-4a24-b09b-a80bd9f47494 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:25.494575 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7fd41d-6945-4a24-b09b-a80bd9f47494 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:25.499535 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc69072c-c82d-4447-96f7-72b8c6c03ff4 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:25.519358 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:25.551740 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9872417-3159-48ff-b6ac-236da09ca897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:25.552438 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9872417-3159-48ff-b6ac-236da09ca897 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:25.552627 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9872417-3159-48ff-b6ac-236da09ca897 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:25.552930 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9872417-3159-48ff-b6ac-236da09ca897 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:25.553691 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9872417-3159-48ff-b6ac-236da09ca897 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:25.553993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9872417-3159-48ff-b6ac-236da09ca897 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:25.556109 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9872417-3159-48ff-b6ac-236da09ca897 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:25.565879 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 112/221] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:25.571634 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9872417-3159-48ff-b6ac-236da09ca897 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:25.590542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7fd41d-6945-4a24-b09b-a80bd9f47494 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:25.593090 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7fd41d-6945-4a24-b09b-a80bd9f47494 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:25.677367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:25.683741 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9872417-3159-48ff-b6ac-236da09ca897 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:25.721446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 110/222] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 16:53:25.729268 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fd41d-6945-4a24-b09b-a80bd9f47494 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:25.740043 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67793ddb-c468-4107-9b7f-b4116c547c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:25.741955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67793ddb-c468-4107-9b7f-b4116c547c2b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:25.742382 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67793ddb-c468-4107-9b7f-b4116c547c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:25.742793 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67793ddb-c468-4107-9b7f-b4116c547c2b None admin] PATH_INFO: `/v3/users/c5ba91cf43124aa98c81468af3f0f4fc` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:25.747714 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67793ddb-c468-4107-9b7f-b4116c547c2b None admin] RBAC: Authorizing `identity:get_user(user_id=c5ba91cf43124aa98c81468af3f0f4fc)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:25.749173 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67793ddb-c468-4107-9b7f-b4116c547c2b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:25.752218 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 113/223] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:25 2022] GET /identity/v3/users/c5ba91cf43124aa98c81468af3f0f4fc => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:25.758194 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67793ddb-c468-4107-9b7f-b4116c547c2b None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:25.838076 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:25.920873 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7fd41d-6945-4a24-b09b-a80bd9f47494 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:25.953505 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67793ddb-c468-4107-9b7f-b4116c547c2b None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:25.993299 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c967a81-67d4-4884-a5ec-84561032c9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:25.994179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c967a81-67d4-4884-a5ec-84561032c9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:25.994458 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c967a81-67d4-4884-a5ec-84561032c9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:25.994731 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c967a81-67d4-4884-a5ec-84561032c9c6 None admin] PATH_INFO: `/v3/projects/4206726f0af54455a4cb03012949ffb5` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:25.998805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c967a81-67d4-4884-a5ec-84561032c9c6 None admin] RBAC: Authorizing `identity:get_project(project_id=4206726f0af54455a4cb03012949ffb5)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:26.000320 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c967a81-67d4-4884-a5ec-84561032c9c6 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:26.005620 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 111/224] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:25 2022] GET /identity/v3/projects/4206726f0af54455a4cb03012949ffb5 => generated 302 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:26.015265 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c967a81-67d4-4884-a5ec-84561032c9c6 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:26.031086 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffe742-c145-4653-a6da-93d09fa3736b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:26.031968 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffe742-c145-4653-a6da-93d09fa3736b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:26.032245 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffe742-c145-4653-a6da-93d09fa3736b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:26.032876 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffe742-c145-4653-a6da-93d09fa3736b None admin] PATH_INFO: `/v3/projects/4206726f0af54455a4cb03012949ffb5` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:26.036139 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffe742-c145-4653-a6da-93d09fa3736b None admin] RBAC: Authorizing `identity:get_project(project_id=4206726f0af54455a4cb03012949ffb5)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:26.038377 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ffe742-c145-4653-a6da-93d09fa3736b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:26.040178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 114/225] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:25 2022] GET /identity/v3/projects/4206726f0af54455a4cb03012949ffb5 => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:26.046210 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffe742-c145-4653-a6da-93d09fa3736b None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:26.089331 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:26.179900 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffe742-c145-4653-a6da-93d09fa3736b None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:26.187147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c967a81-67d4-4884-a5ec-84561032c9c6 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:26.191691 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:26.223677 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff824032-516a-4f66-af0e-6576fc7e341f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:26.224816 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff824032-516a-4f66-af0e-6576fc7e341f None admin] REQUEST_METHOD: `PUT` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:26.225152 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff824032-516a-4f66-af0e-6576fc7e341f None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:26.225498 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff824032-516a-4f66-af0e-6576fc7e341f None admin] PATH_INFO: `/v3/projects/4206726f0af54455a4cb03012949ffb5/users/c5ba91cf43124aa98c81468af3f0f4fc/roles/17bbe7cc1c3641509de5e9dca8534102` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:26.227512 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff824032-516a-4f66-af0e-6576fc7e341f None admin] RBAC: Authorizing `identity:create_grant(project_id=4206726f0af54455a4cb03012949ffb5, user_id=c5ba91cf43124aa98c81468af3f0f4fc, role_id=17bbe7cc1c3641509de5e9dca8534102)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:26.228067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58cccb43-8697-491f-a9f7-385b5a950d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:26.228740 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cccb43-8697-491f-a9f7-385b5a950d69 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:26.228955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff824032-516a-4f66-af0e-6576fc7e341f None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:26.229051 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cccb43-8697-491f-a9f7-385b5a950d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:26.229283 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58cccb43-8697-491f-a9f7-385b5a950d69 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:26.230114 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cccb43-8697-491f-a9f7-385b5a950d69 None admin] RBAC: Adding query filter params (role.id=72cc26a1a66c4fc183b39fad4cb38431, scope.project.id=4206726f0af54455a4cb03012949ffb5, user.id=19eff116df6a4c9a859e361fc6e9d908) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:26.230430 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cccb43-8697-491f-a9f7-385b5a950d69 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:26.232674 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58cccb43-8697-491f-a9f7-385b5a950d69 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:26.239783 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 112/226] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:53:26 2022] GET /identity/v3/role_assignments?user.id=19eff116df6a4c9a859e361fc6e9d908&scope.project.id=4206726f0af54455a4cb03012949ffb5&role.id=72cc26a1a66c4fc183b39fad4cb38431 => generated 259 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:26.241910 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0f2eb-d7c8-4941-933c-68ddffb80e4e None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:26.242208 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0f2eb-d7c8-4941-933c-68ddffb80e4e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:26.242487 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc0f2eb-d7c8-4941-933c-68ddffb80e4e None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:26.244019 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 113/227] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:26.245234 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 115/228] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:53:26 2022] PUT /identity/v3/projects/4206726f0af54455a4cb03012949ffb5/users/c5ba91cf43124aa98c81468af3f0f4fc/roles/17bbe7cc1c3641509de5e9dca8534102 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:53:26.254593 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88f5a0-7f78-447f-9c49-1f78f177beb7 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:26.254944 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88f5a0-7f78-447f-9c49-1f78f177beb7 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:26.255188 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf88f5a0-7f78-447f-9c49-1f78f177beb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:26.292416 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-cf88f5a0-7f78-447f-9c49-1f78f177beb7 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:26.293656 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-cf88f5a0-7f78-447f-9c49-1f78f177beb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:26.391707 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 114/229] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:26.414477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff824032-516a-4f66-af0e-6576fc7e341f None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:26.548514 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff824032-516a-4f66-af0e-6576fc7e341f None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:26.587032 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adeb4f1-ba00-4348-8b49-80524cdcc5c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:26.588705 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adeb4f1-ba00-4348-8b49-80524cdcc5c4 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:26.589207 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adeb4f1-ba00-4348-8b49-80524cdcc5c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:26.589671 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adeb4f1-ba00-4348-8b49-80524cdcc5c4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:26.608128 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adeb4f1-ba00-4348-8b49-80524cdcc5c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:26.616664 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-5adeb4f1-ba00-4348-8b49-80524cdcc5c4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:53:26.616664 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:26.616664 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:26.616664 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:26.616664 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:26.616664 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:26.616664 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:26.616664 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:26.616664 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:26.616664 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:26.616664 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:26.616664 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:26.616664 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:26.616664 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:26.616664 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:26.616664 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:26.617551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:26.617551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:26.617551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:26.617551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:26.617551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:26.617551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:26.617551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:26.617551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:26.617551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:26.617551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:26.617551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:26.617551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:26.617551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:26.617551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:26.617551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:26.617551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:26.618331 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:26.618331 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:26.618331 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:26.618331 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:26.618331 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:26.618331 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:26.618331 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:26.618331 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:26.618331 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:26.618331 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:26.618331 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:53:26.618331 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:26.620710 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 116/230] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 23 16:53:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:26.626579 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf88f5a0-7f78-447f-9c49-1f78f177beb7 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:26.674300 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:26.778710 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf88f5a0-7f78-447f-9c49-1f78f177beb7 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:26.835026 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7503bce-5603-44ae-b393-60e8205208bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:26.835725 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7503bce-5603-44ae-b393-60e8205208bf None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:26.836003 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7503bce-5603-44ae-b393-60e8205208bf None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:26.836274 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7503bce-5603-44ae-b393-60e8205208bf None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:26.837032 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7503bce-5603-44ae-b393-60e8205208bf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:26.837339 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7503bce-5603-44ae-b393-60e8205208bf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:26.838746 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7503bce-5603-44ae-b393-60e8205208bf None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:26.851152 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 115/231] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:53:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:26.856905 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adeb4f1-ba00-4348-8b49-80524cdcc5c4 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:26.920372 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:27.019495 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adeb4f1-ba00-4348-8b49-80524cdcc5c4 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:27.068885 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d735770-9377-45f1-8c7e-ff54abaeba8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:27.069775 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d735770-9377-45f1-8c7e-ff54abaeba8c None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:27.070049 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d735770-9377-45f1-8c7e-ff54abaeba8c None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:27.070318 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d735770-9377-45f1-8c7e-ff54abaeba8c None admin] PATH_INFO: `/v3/users/c5ba91cf43124aa98c81468af3f0f4fc` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:27.071542 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d735770-9377-45f1-8c7e-ff54abaeba8c None admin] RBAC: Authorizing `identity:get_user(user_id=c5ba91cf43124aa98c81468af3f0f4fc)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:27.073007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d735770-9377-45f1-8c7e-ff54abaeba8c None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:27.074958 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 117/232] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:26 2022] GET /identity/v3/users/c5ba91cf43124aa98c81468af3f0f4fc => generated 283 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:27.080007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7503bce-5603-44ae-b393-60e8205208bf None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:27.259870 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:27.265966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7503bce-5603-44ae-b393-60e8205208bf None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:27.301911 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc57792-f000-40cb-9f26-c358fdcf2670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:27.302764 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc57792-f000-40cb-9f26-c358fdcf2670 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:27.303631 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc57792-f000-40cb-9f26-c358fdcf2670 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:27.303918 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc57792-f000-40cb-9f26-c358fdcf2670 None admin] PATH_INFO: `/v3/projects/4206726f0af54455a4cb03012949ffb5` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:27.305168 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc57792-f000-40cb-9f26-c358fdcf2670 None admin] RBAC: Authorizing `identity:get_project(project_id=4206726f0af54455a4cb03012949ffb5)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:27.306638 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc57792-f000-40cb-9f26-c358fdcf2670 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:27.308391 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 116/233] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:27 2022] GET /identity/v3/projects/4206726f0af54455a4cb03012949ffb5 => generated 302 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:27.316384 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d735770-9377-45f1-8c7e-ff54abaeba8c None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:27.484087 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:27.490027 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d735770-9377-45f1-8c7e-ff54abaeba8c None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:27.541426 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429e6922-ec86-43ee-a64f-63e4d70cf0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:27.542336 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e6922-ec86-43ee-a64f-63e4d70cf0e8 None admin] REQUEST_METHOD: `PUT` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:27.542613 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e6922-ec86-43ee-a64f-63e4d70cf0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:27.542885 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429e6922-ec86-43ee-a64f-63e4d70cf0e8 None admin] PATH_INFO: `/v3/projects/4206726f0af54455a4cb03012949ffb5/users/c5ba91cf43124aa98c81468af3f0f4fc/roles/6c30aca17fad4c4fa40b9461c5bda291` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:27.545084 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e6922-ec86-43ee-a64f-63e4d70cf0e8 None admin] RBAC: Authorizing `identity:create_grant(project_id=4206726f0af54455a4cb03012949ffb5, user_id=c5ba91cf43124aa98c81468af3f0f4fc, role_id=6c30aca17fad4c4fa40b9461c5bda291)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:27.546528 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429e6922-ec86-43ee-a64f-63e4d70cf0e8 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:27.579388 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 118/234] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:53:27 2022] PUT /identity/v3/projects/4206726f0af54455a4cb03012949ffb5/users/c5ba91cf43124aa98c81468af3f0f4fc/roles/6c30aca17fad4c4fa40b9461c5bda291 => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:53:27.999264 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1a304-b83e-4274-937c-7e846f3abcab None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:27.999805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1a304-b83e-4274-937c-7e846f3abcab None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:27.999805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1a304-b83e-4274-937c-7e846f3abcab None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:28.001472 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 117/235] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:28.006546 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360f67a-9a2e-4e60-8bfc-94136623b9a9 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:28.006972 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360f67a-9a2e-4e60-8bfc-94136623b9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:28.007146 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4360f67a-9a2e-4e60-8bfc-94136623b9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:28.025651 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-4360f67a-9a2e-4e60-8bfc-94136623b9a9 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:28.026818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-4360f67a-9a2e-4e60-8bfc-94136623b9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:28.095473 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 119/236] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:28.134073 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1a304-b83e-4274-937c-7e846f3abcab None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:28.276755 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1a304-b83e-4274-937c-7e846f3abcab None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:28.318467 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cc9ebe-eb6b-4bfe-9877-24ed298fa19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:28.319299 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc9ebe-eb6b-4bfe-9877-24ed298fa19f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:28.319576 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc9ebe-eb6b-4bfe-9877-24ed298fa19f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:28.319848 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cc9ebe-eb6b-4bfe-9877-24ed298fa19f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:28.336364 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2cc9ebe-eb6b-4bfe-9877-24ed298fa19f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:28.344632 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-c2cc9ebe-eb6b-4bfe-9877-24ed298fa19f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 16:53:28.344632 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:28.344632 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:28.344632 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:28.344632 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:28.344632 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:28.344632 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:28.344632 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:28.344632 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:28.344632 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:28.344632 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:28.344632 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:28.344632 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:28.344632 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:28.344632 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:28.344632 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:28.345513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:28.345513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:28.345513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:28.345513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:28.345513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:28.345513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:28.345513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:28.345513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:28.345513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:28.345513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:28.345513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:28.345513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:28.345513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:28.345513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:28.345513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:28.345513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:28.346293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:28.346293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:28.346293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:28.346293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:28.346293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:28.346293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:28.346293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:28.346293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:28.346293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:28.346293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:28.346293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 16:53:28.346293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:28.346293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 118/237] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:28.352013 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360f67a-9a2e-4e60-8bfc-94136623b9a9 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:28.403297 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b8806-e1e5-4750-b899-5812d057c3e3 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:28.403609 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b8806-e1e5-4750-b899-5812d057c3e3 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:28.403850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7b8806-e1e5-4750-b899-5812d057c3e3 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:28.405235 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 119/238] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:28.416574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939c395-68f0-4d2a-b5e5-11d892202310 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:28.416848 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939c395-68f0-4d2a-b5e5-11d892202310 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:28.417122 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e939c395-68f0-4d2a-b5e5-11d892202310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:28.451651 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:28.452382 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-e939c395-68f0-4d2a-b5e5-11d892202310 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:28.453562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e939c395-68f0-4d2a-b5e5-11d892202310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:28.470599 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4360f67a-9a2e-4e60-8bfc-94136623b9a9 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:28.494438 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:28.549436 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0f628-c746-4d24-97ee-c9aaa46d94dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:28.550041 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0f628-c746-4d24-97ee-c9aaa46d94dd None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:28.550328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0f628-c746-4d24-97ee-c9aaa46d94dd None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:28.550602 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca0f628-c746-4d24-97ee-c9aaa46d94dd None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:28.551399 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0f628-c746-4d24-97ee-c9aaa46d94dd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:28.551710 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0f628-c746-4d24-97ee-c9aaa46d94dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:28.553221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca0f628-c746-4d24-97ee-c9aaa46d94dd None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:28.572767 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 120/239] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:28.578163 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0f628-c746-4d24-97ee-c9aaa46d94dd None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:28.601361 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 120/240] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 16:53:28.653117 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e939c395-68f0-4d2a-b5e5-11d892202310 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:28.735715 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca0f628-c746-4d24-97ee-c9aaa46d94dd None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:28.774943 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11b378b8-440f-485b-bdee-2496fc67b0ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:28.775843 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b378b8-440f-485b-bdee-2496fc67b0ad None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:28.776119 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b378b8-440f-485b-bdee-2496fc67b0ad None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:28.776389 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11b378b8-440f-485b-bdee-2496fc67b0ad None admin] PATH_INFO: `/v3/users/4c307e6b178a459e806805861b3c11c0` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:28.792303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b378b8-440f-485b-bdee-2496fc67b0ad None admin] RBAC: Authorizing `identity:get_user(user_id=4c307e6b178a459e806805861b3c11c0)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:28.793759 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11b378b8-440f-485b-bdee-2496fc67b0ad None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:28.800180 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 121/241] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:28 2022] GET /identity/v3/users/4c307e6b178a459e806805861b3c11c0 => generated 297 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:28.800180 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5920e46-09ae-4bc0-b162-e061f517205f None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:28.800180 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5920e46-09ae-4bc0-b162-e061f517205f None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:28.800180 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5920e46-09ae-4bc0-b162-e061f517205f None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:28.800180 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 122/242] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:28.803854 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5920e46-09ae-4bc0-b162-e061f517205f None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:28.812611 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:28.817375 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e939c395-68f0-4d2a-b5e5-11d892202310 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:28.854727 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3073c0ef-7c0a-4eca-afff-02980a210db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:28.855604 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073c0ef-7c0a-4eca-afff-02980a210db8 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:28.855881 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073c0ef-7c0a-4eca-afff-02980a210db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:28.856152 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3073c0ef-7c0a-4eca-afff-02980a210db8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:28.857618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3073c0ef-7c0a-4eca-afff-02980a210db8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:28.859039 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3073c0ef-7c0a-4eca-afff-02980a210db8 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:28.861190 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 121/243] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:53:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:28.862648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccd6e7-e533-49ec-bef7-1b4bbf1d052c None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:28.862885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccd6e7-e533-49ec-bef7-1b4bbf1d052c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:28.863216 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ccd6e7-e533-49ec-bef7-1b4bbf1d052c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:28.882556 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-30ccd6e7-e533-49ec-bef7-1b4bbf1d052c None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:28.883893 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-30ccd6e7-e533-49ec-bef7-1b4bbf1d052c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:28.899746 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:28.931124 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5920e46-09ae-4bc0-b162-e061f517205f None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:28.964827 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 122/244] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:28.965680 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccd6e7-e533-49ec-bef7-1b4bbf1d052c None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:28.969303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c35fc6-d104-427d-926c-bd82e5bc0027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:28.970305 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c35fc6-d104-427d-926c-bd82e5bc0027 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:28.970777 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c35fc6-d104-427d-926c-bd82e5bc0027 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:28.971310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c35fc6-d104-427d-926c-bd82e5bc0027 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:29.047394 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c35fc6-d104-427d-926c-bd82e5bc0027 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:29.049818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c35fc6-d104-427d-926c-bd82e5bc0027 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:29.059747 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:29.070569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ccd6e7-e533-49ec-bef7-1b4bbf1d052c None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:29.107070 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6394b-2c79-41ee-b7e8-e63176305f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:29.107996 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6394b-2c79-41ee-b7e8-e63176305f30 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:29.108475 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6394b-2c79-41ee-b7e8-e63176305f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:29.108913 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f6394b-2c79-41ee-b7e8-e63176305f30 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:29.109778 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6394b-2c79-41ee-b7e8-e63176305f30 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:29.111442 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f6394b-2c79-41ee-b7e8-e63176305f30 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:29.133847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 123/245] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 23 16:53:28 2022] POST /identity/v3/users => generated 293 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:53:29.135457 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6394b-2c79-41ee-b7e8-e63176305f30 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:29.140290 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:29.183406 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 123/246] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:53:29.188624 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba8250-cfc2-4e7a-a4b0-4cb687b30d31 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:29.189993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba8250-cfc2-4e7a-a4b0-4cb687b30d31 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:29.190437 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccba8250-cfc2-4e7a-a4b0-4cb687b30d31 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:29.196557 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 124/247] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:29.196557 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba8250-cfc2-4e7a-a4b0-4cb687b30d31 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:29.286338 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f6394b-2c79-41ee-b7e8-e63176305f30 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:29.325982 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0202702b-f502-4829-ba9e-78d93ae0fa65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:29.325982 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202702b-f502-4829-ba9e-78d93ae0fa65 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:29.326244 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202702b-f502-4829-ba9e-78d93ae0fa65 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:29.326487 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0202702b-f502-4829-ba9e-78d93ae0fa65 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:29.344518 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccba8250-cfc2-4e7a-a4b0-4cb687b30d31 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:29.346973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0202702b-f502-4829-ba9e-78d93ae0fa65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:29.353691 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:29.356511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-0202702b-f502-4829-ba9e-78d93ae0fa65 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:29.356511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:29.356511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:29.356511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:29.356511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:29.356511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:29.356511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:29.356511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:29.356511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:29.356511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:29.356511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:29.356511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:29.356511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:29.356511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:29.356511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:29.356511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:29.357497 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:29.357497 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:29.357497 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:29.357497 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:29.357497 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:29.357497 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:29.357497 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:29.357497 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:29.357497 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:29.357497 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:29.357497 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:29.357497 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:29.357497 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:29.357497 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:29.357497 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:29.357497 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:29.358278 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:29.358278 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:29.358278 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:29.358278 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:29.358278 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:29.358278 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:29.358278 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:29.358278 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:29.358278 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:29.358278 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:29.358278 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:29.358278 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:29.358278 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 124/248] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:29.359716 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec1ddc-edcd-446f-aee9-bdab5fbba7db None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:29.359716 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec1ddc-edcd-446f-aee9-bdab5fbba7db None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:29.359716 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ec1ddc-edcd-446f-aee9-bdab5fbba7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:29.379466 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-76ec1ddc-edcd-446f-aee9-bdab5fbba7db None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:29.380708 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-76ec1ddc-edcd-446f-aee9-bdab5fbba7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:29.389988 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6e978-3bf9-4c66-ac22-09fd8ef4e3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:29.390983 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6e978-3bf9-4c66-ac22-09fd8ef4e3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:29.391331 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6e978-3bf9-4c66-ac22-09fd8ef4e3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:29.391665 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d6e978-3bf9-4c66-ac22-09fd8ef4e3b9 None admin] PATH_INFO: `/v3/projects/4206726f0af54455a4cb03012949ffb5` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:29.392905 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d6e978-3bf9-4c66-ac22-09fd8ef4e3b9 None admin] RBAC: Authorizing `identity:get_project(project_id=4206726f0af54455a4cb03012949ffb5)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:29.395543 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d6e978-3bf9-4c66-ac22-09fd8ef4e3b9 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:29.397388 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 125/249] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:29 2022] GET /identity/v3/projects/4206726f0af54455a4cb03012949ffb5 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:29.399241 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6e978-3bf9-4c66-ac22-09fd8ef4e3b9 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:29.456829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:29.477411 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 125/250] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:29.478516 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec1ddc-edcd-446f-aee9-bdab5fbba7db None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:29.543909 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d6e978-3bf9-4c66-ac22-09fd8ef4e3b9 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:29.583546 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bb469-de92-4f57-92d4-9357bdf8eaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:29.584232 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bb469-de92-4f57-92d4-9357bdf8eaf8 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:29.584541 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bb469-de92-4f57-92d4-9357bdf8eaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:29.584815 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-838bb469-de92-4f57-92d4-9357bdf8eaf8 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:29.585606 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838bb469-de92-4f57-92d4-9357bdf8eaf8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:29.585857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838bb469-de92-4f57-92d4-9357bdf8eaf8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:29.587290 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-838bb469-de92-4f57-92d4-9357bdf8eaf8 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:29.597069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 126/251] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:29.600170 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bb469-de92-4f57-92d4-9357bdf8eaf8 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:29.603601 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ec1ddc-edcd-446f-aee9-bdab5fbba7db None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:29.640743 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559bf24-dd33-4eae-86bb-c466828436d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:29.641417 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559bf24-dd33-4eae-86bb-c466828436d9 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:29.641692 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559bf24-dd33-4eae-86bb-c466828436d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:29.641969 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d559bf24-dd33-4eae-86bb-c466828436d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:29.642798 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559bf24-dd33-4eae-86bb-c466828436d9 None admin] RBAC: Adding query filter params (role.id=9d4694dd70b94d68acc78c53ca498bbf, scope.project.id=4206726f0af54455a4cb03012949ffb5, user.id=4c307e6b178a459e806805861b3c11c0) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:29.643141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559bf24-dd33-4eae-86bb-c466828436d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:29.643333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:29.645068 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d559bf24-dd33-4eae-86bb-c466828436d9 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:29.651681 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 126/252] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:53:29 2022] GET /identity/v3/role_assignments?user.id=4c307e6b178a459e806805861b3c11c0&scope.project.id=4206726f0af54455a4cb03012949ffb5&role.id=9d4694dd70b94d68acc78c53ca498bbf => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:29.652367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559bf24-dd33-4eae-86bb-c466828436d9 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:29.728775 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-838bb469-de92-4f57-92d4-9357bdf8eaf8 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:29.749552 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:29.773295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b8745-4cbe-4033-9bfb-0934d5b9d0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:29.774353 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b8745-4cbe-4033-9bfb-0934d5b9d0f1 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:29.774770 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b8745-4cbe-4033-9bfb-0934d5b9d0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:29.775300 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548b8745-4cbe-4033-9bfb-0934d5b9d0f1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:29.790471 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548b8745-4cbe-4033-9bfb-0934d5b9d0f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:29.800057 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-548b8745-4cbe-4033-9bfb-0934d5b9d0f1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:53:29.800057 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:29.800057 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:29.800057 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:29.800057 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:29.800057 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:29.800057 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:29.800057 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:29.800057 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:29.800057 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:29.800057 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:29.800057 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:29.800057 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:29.800057 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:29.800057 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:29.800057 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:29.801118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:29.801118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:29.801118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:29.801118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:29.801118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:29.801118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:29.801118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:29.801118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:29.801118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:29.801118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:29.801118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:29.801118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:29.801118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:29.801118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:29.801118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:29.801118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:29.802025 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:29.802025 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:29.802025 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:29.802025 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:29.802025 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:29.802025 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:29.802025 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:29.802025 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:29.802025 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:29.802025 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:29.802025 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:53:29.802025 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:29.805142 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 127/253] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:53:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:29.814203 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b8745-4cbe-4033-9bfb-0934d5b9d0f1 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:29.836565 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d559bf24-dd33-4eae-86bb-c466828436d9 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:29.927926 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22364adc-f0dc-4d87-bc6a-1ac914b93354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:29.928859 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22364adc-f0dc-4d87-bc6a-1ac914b93354 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:29.929143 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22364adc-f0dc-4d87-bc6a-1ac914b93354 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:29.929415 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22364adc-f0dc-4d87-bc6a-1ac914b93354 None admin] PATH_INFO: `/v3/users/19eff116df6a4c9a859e361fc6e9d908` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:29.931650 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22364adc-f0dc-4d87-bc6a-1ac914b93354 None admin] RBAC: Authorizing `identity:get_user(user_id=19eff116df6a4c9a859e361fc6e9d908)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:29.934369 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22364adc-f0dc-4d87-bc6a-1ac914b93354 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:29.936531 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 127/254] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:29 2022] GET /identity/v3/users/19eff116df6a4c9a859e361fc6e9d908 => generated 297 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:29.944068 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22364adc-f0dc-4d87-bc6a-1ac914b93354 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:29.949205 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:29.963036 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548b8745-4cbe-4033-9bfb-0934d5b9d0f1 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:30.066223 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22364adc-f0dc-4d87-bc6a-1ac914b93354 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:30.069831 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01dac2-8a08-4e9d-b6ad-d96ac750662d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:30.071129 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01dac2-8a08-4e9d-b6ad-d96ac750662d None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:30.071627 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01dac2-8a08-4e9d-b6ad-d96ac750662d None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:30.072099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff01dac2-8a08-4e9d-b6ad-d96ac750662d None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:30.073042 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01dac2-8a08-4e9d-b6ad-d96ac750662d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:30.073567 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01dac2-8a08-4e9d-b6ad-d96ac750662d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:30.079409 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff01dac2-8a08-4e9d-b6ad-d96ac750662d None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:30.093384 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 128/255] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:30.097323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:30.099640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01dac2-8a08-4e9d-b6ad-d96ac750662d None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:30.107268 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4200d5-e76c-497d-9475-80e5c13eb91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:30.108157 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4200d5-e76c-497d-9475-80e5c13eb91c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:30.108441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4200d5-e76c-497d-9475-80e5c13eb91c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:30.108714 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4200d5-e76c-497d-9475-80e5c13eb91c None admin] PATH_INFO: `/v3/projects/4206726f0af54455a4cb03012949ffb5` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:30.109972 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4200d5-e76c-497d-9475-80e5c13eb91c None admin] RBAC: Authorizing `identity:get_project(project_id=4206726f0af54455a4cb03012949ffb5)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:30.111523 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4200d5-e76c-497d-9475-80e5c13eb91c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:30.113414 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 128/256] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:29 2022] GET /identity/v3/projects/4206726f0af54455a4cb03012949ffb5 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:30.119223 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4200d5-e76c-497d-9475-80e5c13eb91c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:30.268831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4200d5-e76c-497d-9475-80e5c13eb91c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:30.320605 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956bd9a6-3a71-4fd4-9692-92cfd0409cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:30.321564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bd9a6-3a71-4fd4-9692-92cfd0409cf1 None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:30.321850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bd9a6-3a71-4fd4-9692-92cfd0409cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:30.322126 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bd9a6-3a71-4fd4-9692-92cfd0409cf1 None admin] PATH_INFO: `/v3/projects/4206726f0af54455a4cb03012949ffb5/users/19eff116df6a4c9a859e361fc6e9d908/roles/72cc26a1a66c4fc183b39fad4cb38431` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:30.324138 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956bd9a6-3a71-4fd4-9692-92cfd0409cf1 None admin] RBAC: Authorizing `identity:create_grant(project_id=4206726f0af54455a4cb03012949ffb5, user_id=19eff116df6a4c9a859e361fc6e9d908, role_id=72cc26a1a66c4fc183b39fad4cb38431)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:30.325613 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956bd9a6-3a71-4fd4-9692-92cfd0409cf1 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:30.337920 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff01dac2-8a08-4e9d-b6ad-d96ac750662d None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:30.367313 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 129/257] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:53:30 2022] PUT /identity/v3/projects/4206726f0af54455a4cb03012949ffb5/users/19eff116df6a4c9a859e361fc6e9d908/roles/72cc26a1a66c4fc183b39fad4cb38431 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 16:53:30.426417 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc06fff-cf9f-4185-ba2b-9764d7e2b21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:30.429487 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc06fff-cf9f-4185-ba2b-9764d7e2b21a None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:30.431809 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc06fff-cf9f-4185-ba2b-9764d7e2b21a None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:30.432334 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc06fff-cf9f-4185-ba2b-9764d7e2b21a None admin] PATH_INFO: `/v3/users/c5ba91cf43124aa98c81468af3f0f4fc` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:30.436234 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc06fff-cf9f-4185-ba2b-9764d7e2b21a None admin] RBAC: Authorizing `identity:get_user(user_id=c5ba91cf43124aa98c81468af3f0f4fc)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:30.438014 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc06fff-cf9f-4185-ba2b-9764d7e2b21a None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:30.448000 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956bd9a6-3a71-4fd4-9692-92cfd0409cf1 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:30.448610 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 129/258] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:30 2022] GET /identity/v3/users/c5ba91cf43124aa98c81468af3f0f4fc => generated 283 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:30.563156 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:30.644574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956bd9a6-3a71-4fd4-9692-92cfd0409cf1 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:30.695663 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528035b-a5ca-452e-84a8-1600113614ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:30.696342 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528035b-a5ca-452e-84a8-1600113614ce None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:30.696623 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528035b-a5ca-452e-84a8-1600113614ce None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:30.696899 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6528035b-a5ca-452e-84a8-1600113614ce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:30.775822 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6528035b-a5ca-452e-84a8-1600113614ce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:30.777303 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6528035b-a5ca-452e-84a8-1600113614ce None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:30.836544 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83040c42-d447-4930-82c3-f5534c0d436d None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:30.836864 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83040c42-d447-4930-82c3-f5534c0d436d None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:30.837103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83040c42-d447-4930-82c3-f5534c0d436d None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:30.838630 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 130/259] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:30.844164 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04af72-00b5-4a80-8218-e7a8b6652ac6 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:30.844453 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04af72-00b5-4a80-8218-e7a8b6652ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:30.844728 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b04af72-00b5-4a80-8218-e7a8b6652ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:30.850513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:30.896023 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-7b04af72-00b5-4a80-8218-e7a8b6652ac6 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:30.897849 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-7b04af72-00b5-4a80-8218-e7a8b6652ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:30.899374 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 130/260] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:53:30.910601 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528035b-a5ca-452e-84a8-1600113614ce None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:30.967935 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:31.062528 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6528035b-a5ca-452e-84a8-1600113614ce None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:31.121427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2510793-ae01-4439-8d19-15435a5ae6a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:31.122288 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2510793-ae01-4439-8d19-15435a5ae6a5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:31.122725 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2510793-ae01-4439-8d19-15435a5ae6a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:31.123170 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2510793-ae01-4439-8d19-15435a5ae6a5 None admin] PATH_INFO: `/v3/projects/4206726f0af54455a4cb03012949ffb5` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:31.127638 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2510793-ae01-4439-8d19-15435a5ae6a5 None admin] RBAC: Authorizing `identity:get_project(project_id=4206726f0af54455a4cb03012949ffb5)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:31.129115 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2510793-ae01-4439-8d19-15435a5ae6a5 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:31.131470 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 131/261] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:30 2022] GET /identity/v3/projects/4206726f0af54455a4cb03012949ffb5 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:31.134767 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 131/262] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:31.137760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2510793-ae01-4439-8d19-15435a5ae6a5 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:31.157949 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04af72-00b5-4a80-8218-e7a8b6652ac6 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:31.192676 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:31.331185 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b04af72-00b5-4a80-8218-e7a8b6652ac6 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:31.392588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b75d20-26aa-4895-a759-1a6f309edc69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:31.393431 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b75d20-26aa-4895-a759-1a6f309edc69 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:31.393715 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b75d20-26aa-4895-a759-1a6f309edc69 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:31.393985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b75d20-26aa-4895-a759-1a6f309edc69 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:31.413917 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b75d20-26aa-4895-a759-1a6f309edc69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:31.422850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-f9b75d20-26aa-4895-a759-1a6f309edc69 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:53:31.422850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:31.422850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:31.422850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:31.422850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:31.422850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:31.422850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:31.422850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:31.422850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:31.422850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:31.422850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:31.422850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:31.422850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:31.422850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:31.422850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:31.422850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:31.423826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:31.423826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:31.423826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:31.423826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:31.423826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:31.423826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:31.423826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:31.423826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:31.423826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:31.423826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:31.423826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:31.423826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:31.423826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:31.423826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:31.423826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:31.423826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:31.424696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:31.424696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:31.424696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:31.424696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:31.424696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:31.424696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:31.424696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:31.424696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:31.424696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:31.424696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:31.424696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:53:31.424696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:31.431802 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2510793-ae01-4439-8d19-15435a5ae6a5 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:31.432388 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 132/263] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 23 16:53:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:53:31.439012 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b75d20-26aa-4895-a759-1a6f309edc69 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:31.466405 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:31.527503 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7470212-c376-45e5-83b6-0ca246285a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:31.528187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7470212-c376-45e5-83b6-0ca246285a9f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:31.528466 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7470212-c376-45e5-83b6-0ca246285a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:31.528737 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7470212-c376-45e5-83b6-0ca246285a9f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:31.529582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7470212-c376-45e5-83b6-0ca246285a9f None admin] RBAC: Adding query filter params (role.id=17bbe7cc1c3641509de5e9dca8534102, scope.project.id=4206726f0af54455a4cb03012949ffb5, user.id=c5ba91cf43124aa98c81468af3f0f4fc) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:31.529891 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7470212-c376-45e5-83b6-0ca246285a9f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:31.531423 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7470212-c376-45e5-83b6-0ca246285a9f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:31.549071 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 132/264] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:53:31 2022] GET /identity/v3/role_assignments?user.id=c5ba91cf43124aa98c81468af3f0f4fc&scope.project.id=4206726f0af54455a4cb03012949ffb5&role.id=17bbe7cc1c3641509de5e9dca8534102 => generated 608 bytes in 411 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:31.616297 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b75d20-26aa-4895-a759-1a6f309edc69 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:31.668645 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4840cef-4ef5-4c88-8a9d-6420ec2810b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:31.669264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4840cef-4ef5-4c88-8a9d-6420ec2810b9 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:31.669546 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4840cef-4ef5-4c88-8a9d-6420ec2810b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:31.669820 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4840cef-4ef5-4c88-8a9d-6420ec2810b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:31.670593 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4840cef-4ef5-4c88-8a9d-6420ec2810b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:31.670901 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4840cef-4ef5-4c88-8a9d-6420ec2810b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:31.672353 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4840cef-4ef5-4c88-8a9d-6420ec2810b9 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:31.687367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 133/265] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:53:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:31.688406 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7470212-c376-45e5-83b6-0ca246285a9f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:31.808456 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:31.826251 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7470212-c376-45e5-83b6-0ca246285a9f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:31.874789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0842d86-60f1-47bc-856d-dc3281b19804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:31.875709 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0842d86-60f1-47bc-856d-dc3281b19804 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:31.875987 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0842d86-60f1-47bc-856d-dc3281b19804 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:31.876252 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0842d86-60f1-47bc-856d-dc3281b19804 None admin] PATH_INFO: `/v3/users/c5ba91cf43124aa98c81468af3f0f4fc` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:31.883679 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0842d86-60f1-47bc-856d-dc3281b19804 None admin] RBAC: Authorizing `identity:get_user(user_id=c5ba91cf43124aa98c81468af3f0f4fc)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:31.885486 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0842d86-60f1-47bc-856d-dc3281b19804 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:31.887555 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 133/266] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:31 2022] GET /identity/v3/users/c5ba91cf43124aa98c81468af3f0f4fc => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:31.892974 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4840cef-4ef5-4c88-8a9d-6420ec2810b9 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:31.980099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:32.003302 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4840cef-4ef5-4c88-8a9d-6420ec2810b9 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:32.040971 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38c457-f350-4a9d-ab99-27d7e535b38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:32.041635 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38c457-f350-4a9d-ab99-27d7e535b38b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:32.042267 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38c457-f350-4a9d-ab99-27d7e535b38b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:32.042267 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d38c457-f350-4a9d-ab99-27d7e535b38b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:32.113313 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38c457-f350-4a9d-ab99-27d7e535b38b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:32.114795 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d38c457-f350-4a9d-ab99-27d7e535b38b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:32.196107 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:32.221420 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 134/267] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:53:32.227630 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0842d86-60f1-47bc-856d-dc3281b19804 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:32.347508 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0842d86-60f1-47bc-856d-dc3281b19804 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:32.386764 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aaf4ece-f1f6-40ac-a21a-08bf4d31233d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:32.387997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf4ece-f1f6-40ac-a21a-08bf4d31233d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:32.388422 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf4ece-f1f6-40ac-a21a-08bf4d31233d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:32.388835 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf4ece-f1f6-40ac-a21a-08bf4d31233d None admin] PATH_INFO: `/v3/projects/4206726f0af54455a4cb03012949ffb5` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:32.395662 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf4ece-f1f6-40ac-a21a-08bf4d31233d None admin] RBAC: Authorizing `identity:get_project(project_id=4206726f0af54455a4cb03012949ffb5)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:32.397460 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aaf4ece-f1f6-40ac-a21a-08bf4d31233d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:32.399851 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 134/268] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:32 2022] GET /identity/v3/projects/4206726f0af54455a4cb03012949ffb5 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:32.405496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38c457-f350-4a9d-ab99-27d7e535b38b None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:32.442996 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22feb8-0fbf-4bda-8630-1402a2cfdafc None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:32.443567 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22feb8-0fbf-4bda-8630-1402a2cfdafc None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:32.443987 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb22feb8-0fbf-4bda-8630-1402a2cfdafc None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:32.445734 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 135/269] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:32.451297 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b8e96-91dd-4aa2-bc0c-247ee83143c7 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:32.451750 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b8e96-91dd-4aa2-bc0c-247ee83143c7 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:32.452187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af9b8e96-91dd-4aa2-bc0c-247ee83143c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:32.499951 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-af9b8e96-91dd-4aa2-bc0c-247ee83143c7 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:32.501383 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-af9b8e96-91dd-4aa2-bc0c-247ee83143c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:32.533926 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:32.536534 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d38c457-f350-4a9d-ab99-27d7e535b38b None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:32.573311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd976be2-9bdf-4cfb-b379-5826e9055d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:32.574217 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd976be2-9bdf-4cfb-b379-5826e9055d4f None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:32.574640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd976be2-9bdf-4cfb-b379-5826e9055d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:32.575099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd976be2-9bdf-4cfb-b379-5826e9055d4f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:32.576077 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd976be2-9bdf-4cfb-b379-5826e9055d4f None admin] RBAC: Adding query filter params (role.id=6c30aca17fad4c4fa40b9461c5bda291, scope.project.id=4206726f0af54455a4cb03012949ffb5, user.id=c5ba91cf43124aa98c81468af3f0f4fc) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:32.576559 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd976be2-9bdf-4cfb-b379-5826e9055d4f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:32.578145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd976be2-9bdf-4cfb-b379-5826e9055d4f None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:32.585518 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 135/270] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:53:32 2022] GET /identity/v3/role_assignments?user.id=c5ba91cf43124aa98c81468af3f0f4fc&scope.project.id=4206726f0af54455a4cb03012949ffb5&role.id=6c30aca17fad4c4fa40b9461c5bda291 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:32.640420 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 136/271] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 16:53:32.685070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd976be2-9bdf-4cfb-b379-5826e9055d4f None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:32.739902 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:32.791844 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a3dc2-18ef-4253-9b1e-02ad86cb384c None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:32.792167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a3dc2-18ef-4253-9b1e-02ad86cb384c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:32.792399 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1a3dc2-18ef-4253-9b1e-02ad86cb384c None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:32.793792 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 137/272] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:53:32.799038 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d013e-968e-4f58-ae52-cbb7dec68be0 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:32.799323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d013e-968e-4f58-ae52-cbb7dec68be0 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:32.799597 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f86d013e-968e-4f58-ae52-cbb7dec68be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:32.820110 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-f86d013e-968e-4f58-ae52-cbb7dec68be0 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:32.821804 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f86d013e-968e-4f58-ae52-cbb7dec68be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:32.846947 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd976be2-9bdf-4cfb-b379-5826e9055d4f None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:32.907576 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 138/273] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:32.932249 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d013e-968e-4f58-ae52-cbb7dec68be0 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:32.935940 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541917d8-e7cb-4ac2-a924-bb01401a0954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:32.936802 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541917d8-e7cb-4ac2-a924-bb01401a0954 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:32.937069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541917d8-e7cb-4ac2-a924-bb01401a0954 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:32.937341 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541917d8-e7cb-4ac2-a924-bb01401a0954 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:32.939803 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541917d8-e7cb-4ac2-a924-bb01401a0954 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:32.941267 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541917d8-e7cb-4ac2-a924-bb01401a0954 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:32.943477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 136/274] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:53:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:32.950117 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541917d8-e7cb-4ac2-a924-bb01401a0954 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:33.036209 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:33.065615 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f86d013e-968e-4f58-ae52-cbb7dec68be0 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:33.073326 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541917d8-e7cb-4ac2-a924-bb01401a0954 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:33.084942 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:33.114555 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1aec1a-b53f-4c06-a2c2-4a402eae0b93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:33.115511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1aec1a-b53f-4c06-a2c2-4a402eae0b93 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:33.115783 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1aec1a-b53f-4c06-a2c2-4a402eae0b93 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:33.116176 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1aec1a-b53f-4c06-a2c2-4a402eae0b93 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:33.118372 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1259a0bc-5d25-41dc-8f51-8649ec66eadd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:33.119358 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259a0bc-5d25-41dc-8f51-8649ec66eadd None admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:33.119807 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259a0bc-5d25-41dc-8f51-8649ec66eadd None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:33.120224 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1259a0bc-5d25-41dc-8f51-8649ec66eadd None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:33.121078 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1259a0bc-5d25-41dc-8f51-8649ec66eadd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:33.122667 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1259a0bc-5d25-41dc-8f51-8649ec66eadd None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:33.129182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1aec1a-b53f-4c06-a2c2-4a402eae0b93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:33.139690 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-0f1aec1a-b53f-4c06-a2c2-4a402eae0b93 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:33.139690 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:33.139690 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:33.139690 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:33.139690 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:33.139690 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:33.139690 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:33.139690 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:33.139690 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:33.139690 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:33.139690 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:33.139690 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:33.139690 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:33.139690 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:33.139690 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:33.139690 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:33.140885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:33.140885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:33.140885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:33.140885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:33.140885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:33.140885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:33.140885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:33.140885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:33.140885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:33.140885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:33.140885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:33.140885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:33.140885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:33.140885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:33.140885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:33.140885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:33.141668 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:33.141668 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:33.141668 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:33.141668 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:33.141668 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:33.141668 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:33.141668 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:33.141668 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:33.141668 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:33.141668 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:33.141668 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:33.141668 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:33.141668 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 139/275] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:53:33.144088 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca58c77-44be-4de7-a0ea-de468b628c83 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:33.144373 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca58c77-44be-4de7-a0ea-de468b628c83 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:33.144647 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca58c77-44be-4de7-a0ea-de468b628c83 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:33.145971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 140/276] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:33.151021 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca58c77-44be-4de7-a0ea-de468b628c83 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:33.153617 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 137/277] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 23 16:53:32 2022] POST /identity/v3/users => generated 293 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:53:33.155567 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cf52e-f21b-402d-af60-24fd12cc1fc1 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:33.155859 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cf52e-f21b-402d-af60-24fd12cc1fc1 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:33.156135 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2cf52e-f21b-402d-af60-24fd12cc1fc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:33.175922 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-ef2cf52e-f21b-402d-af60-24fd12cc1fc1 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:33.177287 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-ef2cf52e-f21b-402d-af60-24fd12cc1fc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:33.261973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 138/278] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:33.284831 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cf52e-f21b-402d-af60-24fd12cc1fc1 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:33.294622 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca58c77-44be-4de7-a0ea-de468b628c83 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:33.334755 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb9a80-3fc0-4d02-8812-f213e459773b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:33.335841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb9a80-3fc0-4d02-8812-f213e459773b None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:33.336416 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb9a80-3fc0-4d02-8812-f213e459773b None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:33.337009 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8eb9a80-3fc0-4d02-8812-f213e459773b None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:33.338012 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb9a80-3fc0-4d02-8812-f213e459773b None admin] RBAC: Adding query filter params (name=member) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:33.338556 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb9a80-3fc0-4d02-8812-f213e459773b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:33.340274 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8eb9a80-3fc0-4d02-8812-f213e459773b None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:33.351446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 141/279] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:33.357589 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb9a80-3fc0-4d02-8812-f213e459773b None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:33.363129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:33.417235 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:33.424373 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2cf52e-f21b-402d-af60-24fd12cc1fc1 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:33.466030 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e09eb-a0e6-4ad0-a1cc-b86c592caddb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:33.466773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e09eb-a0e6-4ad0-a1cc-b86c592caddb None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:33.467093 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e09eb-a0e6-4ad0-a1cc-b86c592caddb None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:33.467473 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578e09eb-a0e6-4ad0-a1cc-b86c592caddb None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:33.481094 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578e09eb-a0e6-4ad0-a1cc-b86c592caddb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:33.481746 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8eb9a80-3fc0-4d02-8812-f213e459773b None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:33.489252 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-578e09eb-a0e6-4ad0-a1cc-b86c592caddb None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 16:53:33.489252 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:33.489252 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:33.489252 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:33.489252 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:33.489252 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:33.489252 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:33.489252 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:33.489252 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:33.489252 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:33.489252 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:33.489252 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:33.489252 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:33.489252 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:33.489252 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:33.489252 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:33.490287 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:33.490287 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:33.490287 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:33.490287 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:33.490287 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:33.490287 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:33.490287 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:33.490287 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:33.490287 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:33.490287 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:33.490287 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:33.490287 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:33.490287 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:33.490287 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:33.490287 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:33.490287 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:33.491131 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:33.491131 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:33.491131 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:33.491131 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:33.491131 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:33.491131 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:33.491131 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:33.491131 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:33.491131 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:33.491131 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:33.491131 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 16:53:33.491131 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:33.491131 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 139/280] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:53:33.495679 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e09eb-a0e6-4ad0-a1cc-b86c592caddb None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:33.529369 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d22f4-9bef-4ce3-993c-b7b74f01edef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:33.530623 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d22f4-9bef-4ce3-993c-b7b74f01edef None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:33.531175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d22f4-9bef-4ce3-993c-b7b74f01edef None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:33.531524 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1d22f4-9bef-4ce3-993c-b7b74f01edef None admin] PATH_INFO: `/v3/users/c1223f9cfdbf4ae0a971ef90f629493c` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:33.547477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d22f4-9bef-4ce3-993c-b7b74f01edef None admin] RBAC: Authorizing `identity:get_user(user_id=c1223f9cfdbf4ae0a971ef90f629493c)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:33.549198 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1d22f4-9bef-4ce3-993c-b7b74f01edef None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:33.551431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 142/281] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:33 2022] GET /identity/v3/users/c1223f9cfdbf4ae0a971ef90f629493c => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:33.558270 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d22f4-9bef-4ce3-993c-b7b74f01edef None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:33.620927 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578e09eb-a0e6-4ad0-a1cc-b86c592caddb None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:33.654650 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:33.667193 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ba606-02a0-4229-98ec-c07fe5956a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:33.667979 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ba606-02a0-4229-98ec-c07fe5956a1b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:33.668264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ba606-02a0-4229-98ec-c07fe5956a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:33.668537 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ba606-02a0-4229-98ec-c07fe5956a1b None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:33.669311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ba606-02a0-4229-98ec-c07fe5956a1b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:33.669624 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ba606-02a0-4229-98ec-c07fe5956a1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:33.671060 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ba606-02a0-4229-98ec-c07fe5956a1b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:33.680610 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 140/282] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:33.682055 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c05c47-42c7-4100-aa5d-fee1e962a82c None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:33.682352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c05c47-42c7-4100-aa5d-fee1e962a82c None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:33.682610 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c05c47-42c7-4100-aa5d-fee1e962a82c None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:33.686947 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 141/283] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:33.687820 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c05c47-42c7-4100-aa5d-fee1e962a82c None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:33.707487 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1d22f4-9bef-4ce3-993c-b7b74f01edef None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:33.715290 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:33.746247 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd3a265-03b2-4a35-b9b4-39d744003b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:33.747203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3a265-03b2-4a35-b9b4-39d744003b0d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:33.747510 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3a265-03b2-4a35-b9b4-39d744003b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:33.747789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd3a265-03b2-4a35-b9b4-39d744003b0d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:33.748889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd3a265-03b2-4a35-b9b4-39d744003b0d None admin] RBAC: Adding query filter params (role.id=72cc26a1a66c4fc183b39fad4cb38431, user.id=c1223f9cfdbf4ae0a971ef90f629493c, scope.system=all) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:33.749215 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd3a265-03b2-4a35-b9b4-39d744003b0d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:33.750831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd3a265-03b2-4a35-b9b4-39d744003b0d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:33.757249 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 143/284] 10.222.0.13 () {56 vars in 1308 bytes} [Mon May 23 16:53:33 2022] GET /identity/v3/role_assignments?user.id=c1223f9cfdbf4ae0a971ef90f629493c&scope.system=all&role.id=72cc26a1a66c4fc183b39fad4cb38431 => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:33.763178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08714b5-d52b-4073-8152-bcde470f1ba1 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:33.763178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08714b5-d52b-4073-8152-bcde470f1ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:33.763178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08714b5-d52b-4073-8152-bcde470f1ba1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:33.778973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-b08714b5-d52b-4073-8152-bcde470f1ba1 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:33.780242 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-b08714b5-d52b-4073-8152-bcde470f1ba1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:33.840782 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c05c47-42c7-4100-aa5d-fee1e962a82c None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:33.911187 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614230f-8ac0-4ba4-b7cb-f09d4ea242fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:33.912752 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614230f-8ac0-4ba4-b7cb-f09d4ea242fe None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:33.913230 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614230f-8ac0-4ba4-b7cb-f09d4ea242fe None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:33.913713 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7614230f-8ac0-4ba4-b7cb-f09d4ea242fe None admin] PATH_INFO: `/v3/users/4c307e6b178a459e806805861b3c11c0` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:33.915663 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614230f-8ac0-4ba4-b7cb-f09d4ea242fe None admin] RBAC: Authorizing `identity:get_user(user_id=4c307e6b178a459e806805861b3c11c0)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:33.917417 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7614230f-8ac0-4ba4-b7cb-f09d4ea242fe None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:33.927091 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 142/285] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:33 2022] GET /identity/v3/users/4c307e6b178a459e806805861b3c11c0 => generated 297 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:33.929899 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614230f-8ac0-4ba4-b7cb-f09d4ea242fe None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:33.952629 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 144/286] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:33.996977 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08714b5-d52b-4073-8152-bcde470f1ba1 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:34.058856 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:34.074367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:34.101659 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7614230f-8ac0-4ba4-b7cb-f09d4ea242fe None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:34.128537 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08714b5-d52b-4073-8152-bcde470f1ba1 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:34.142961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cbb99-d391-4fda-9da6-e1da694db466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:34.142961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cbb99-d391-4fda-9da6-e1da694db466 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:34.142961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cbb99-d391-4fda-9da6-e1da694db466 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:34.142961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33cbb99-d391-4fda-9da6-e1da694db466 None admin] PATH_INFO: `/v3/projects/4206726f0af54455a4cb03012949ffb5` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:34.144492 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33cbb99-d391-4fda-9da6-e1da694db466 None admin] RBAC: Authorizing `identity:get_project(project_id=4206726f0af54455a4cb03012949ffb5)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:34.150942 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33cbb99-d391-4fda-9da6-e1da694db466 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:34.153029 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 143/287] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:33 2022] GET /identity/v3/projects/4206726f0af54455a4cb03012949ffb5 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:34.155111 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cbb99-d391-4fda-9da6-e1da694db466 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:34.213002 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803b5ac-2ccc-41d0-bfab-55fe7d10726d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:34.213814 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803b5ac-2ccc-41d0-bfab-55fe7d10726d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:34.214099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803b5ac-2ccc-41d0-bfab-55fe7d10726d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:34.214373 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2803b5ac-2ccc-41d0-bfab-55fe7d10726d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:34.247967 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2803b5ac-2ccc-41d0-bfab-55fe7d10726d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:34.255760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-2803b5ac-2ccc-41d0-bfab-55fe7d10726d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:34.255760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:34.255760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:34.255760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:34.255760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:34.255760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:34.255760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:34.255760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:34.255760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:34.255760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:34.255760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:34.255760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:34.255760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:34.255760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:34.255760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:34.255760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:34.256653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:34.256653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:34.256653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:34.256653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:34.256653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:34.256653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:34.256653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:34.256653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:34.256653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:34.256653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:34.256653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:34.256653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:34.256653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:34.256653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:34.256653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:34.256653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:34.257849 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:34.257849 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:34.257849 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:34.257849 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:34.257849 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:34.257849 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:34.257849 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:34.257849 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:34.257849 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:34.257849 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:34.257849 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:34.257849 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:34.257849 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 145/288] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 261 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:53:34.267956 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803b5ac-2ccc-41d0-bfab-55fe7d10726d None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:34.283962 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33cbb99-d391-4fda-9da6-e1da694db466 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:34.355338 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8f02b-2415-4715-9883-7ad3998bd576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:34.356604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8f02b-2415-4715-9883-7ad3998bd576 None admin] REQUEST_METHOD: `PUT` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:34.359421 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8f02b-2415-4715-9883-7ad3998bd576 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:34.359896 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e8f02b-2415-4715-9883-7ad3998bd576 None admin] PATH_INFO: `/v3/projects/4206726f0af54455a4cb03012949ffb5/users/4c307e6b178a459e806805861b3c11c0/roles/9d4694dd70b94d68acc78c53ca498bbf` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:34.363537 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e8f02b-2415-4715-9883-7ad3998bd576 None admin] RBAC: Authorizing `identity:create_grant(project_id=4206726f0af54455a4cb03012949ffb5, user_id=4c307e6b178a459e806805861b3c11c0, role_id=9d4694dd70b94d68acc78c53ca498bbf)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:34.367637 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e8f02b-2415-4715-9883-7ad3998bd576 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:34.396248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 144/289] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:53:34 2022] PUT /identity/v3/projects/4206726f0af54455a4cb03012949ffb5/users/4c307e6b178a459e806805861b3c11c0/roles/9d4694dd70b94d68acc78c53ca498bbf => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:53:34.452248 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2803b5ac-2ccc-41d0-bfab-55fe7d10726d None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:34.474744 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:34.491109 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf921a0d-570f-4428-84d3-1f395dd5cc23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:34.491896 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf921a0d-570f-4428-84d3-1f395dd5cc23 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:34.492175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf921a0d-570f-4428-84d3-1f395dd5cc23 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:34.492443 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf921a0d-570f-4428-84d3-1f395dd5cc23 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:34.493193 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf921a0d-570f-4428-84d3-1f395dd5cc23 None admin] RBAC: Adding query filter params (name=member) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:34.493496 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf921a0d-570f-4428-84d3-1f395dd5cc23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:34.494902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf921a0d-570f-4428-84d3-1f395dd5cc23 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:34.506405 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 146/290] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:34.512141 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8f02b-2415-4715-9883-7ad3998bd576 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:34.540158 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:34.673896 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e8f02b-2415-4715-9883-7ad3998bd576 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:34.716992 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9a9fc-af8e-48af-a344-2d12e8c4eb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:34.718264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9a9fc-af8e-48af-a344-2d12e8c4eb4e None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:34.718766 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9a9fc-af8e-48af-a344-2d12e8c4eb4e None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:34.719282 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9a9fc-af8e-48af-a344-2d12e8c4eb4e None admin] PATH_INFO: `/v3/users/19eff116df6a4c9a859e361fc6e9d908` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:34.720718 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab9a9fc-af8e-48af-a344-2d12e8c4eb4e None admin] RBAC: Authorizing `identity:get_user(user_id=19eff116df6a4c9a859e361fc6e9d908)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:34.722374 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab9a9fc-af8e-48af-a344-2d12e8c4eb4e None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:34.724336 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 145/291] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:34 2022] GET /identity/v3/users/19eff116df6a4c9a859e361fc6e9d908 => generated 297 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:34.730070 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf921a0d-570f-4428-84d3-1f395dd5cc23 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:34.859092 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf921a0d-570f-4428-84d3-1f395dd5cc23 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:34.904745 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06d44a-4d39-401d-bbbd-1998e4cb15b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:34.905811 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06d44a-4d39-401d-bbbd-1998e4cb15b1 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:34.906349 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06d44a-4d39-401d-bbbd-1998e4cb15b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:34.906830 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a06d44a-4d39-401d-bbbd-1998e4cb15b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:34.975996 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:35.005043 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06d44a-4d39-401d-bbbd-1998e4cb15b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:35.006690 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a06d44a-4d39-401d-bbbd-1998e4cb15b1 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:35.116271 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 147/292] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 16:53:35.124017 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9a9fc-af8e-48af-a344-2d12e8c4eb4e None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:35.212178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:35.232571 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9a9fc-af8e-48af-a344-2d12e8c4eb4e None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:35.267476 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea8ebcf-a7da-42b9-8f14-08db3a165f2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:35.268357 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8ebcf-a7da-42b9-8f14-08db3a165f2e None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:35.268638 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8ebcf-a7da-42b9-8f14-08db3a165f2e None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:35.268908 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea8ebcf-a7da-42b9-8f14-08db3a165f2e None admin] PATH_INFO: `/v3/projects/4206726f0af54455a4cb03012949ffb5` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:35.270148 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea8ebcf-a7da-42b9-8f14-08db3a165f2e None admin] RBAC: Authorizing `identity:get_project(project_id=4206726f0af54455a4cb03012949ffb5)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:35.271765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea8ebcf-a7da-42b9-8f14-08db3a165f2e None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:35.273482 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 146/293] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:35 2022] GET /identity/v3/projects/4206726f0af54455a4cb03012949ffb5 => generated 302 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:35.279337 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06d44a-4d39-401d-bbbd-1998e4cb15b1 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:35.382639 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:35.405088 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a06d44a-4d39-401d-bbbd-1998e4cb15b1 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:35.455682 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae4fba0-923d-4445-a5ed-e2366f8e79b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:35.456349 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4fba0-923d-4445-a5ed-e2366f8e79b7 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:35.456640 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4fba0-923d-4445-a5ed-e2366f8e79b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:35.456915 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae4fba0-923d-4445-a5ed-e2366f8e79b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:35.457759 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4fba0-923d-4445-a5ed-e2366f8e79b7 None admin] RBAC: Adding query filter params (role.id=72cc26a1a66c4fc183b39fad4cb38431, scope.project.id=4206726f0af54455a4cb03012949ffb5, user.id=19eff116df6a4c9a859e361fc6e9d908) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:35.458074 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4fba0-923d-4445-a5ed-e2366f8e79b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:35.459676 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eae4fba0-923d-4445-a5ed-e2366f8e79b7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:35.471389 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 148/294] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:53:35 2022] GET /identity/v3/role_assignments?user.id=19eff116df6a4c9a859e361fc6e9d908&scope.project.id=4206726f0af54455a4cb03012949ffb5&role.id=72cc26a1a66c4fc183b39fad4cb38431 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:35.911651 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c26c7d-fb31-42cf-aed1-ab2043de6e35 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:35.912276 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c26c7d-fb31-42cf-aed1-ab2043de6e35 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:35.912276 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75c26c7d-fb31-42cf-aed1-ab2043de6e35 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:35.913913 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 147/295] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:35.919640 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55b9ee-7322-4e16-b272-6a9290ee6eb7 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:35.919932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55b9ee-7322-4e16-b272-6a9290ee6eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:35.920213 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b55b9ee-7322-4e16-b272-6a9290ee6eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:35.944043 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-1b55b9ee-7322-4e16-b272-6a9290ee6eb7 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:35.945225 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-1b55b9ee-7322-4e16-b272-6a9290ee6eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:35.999436 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beab104-1aab-41a4-9836-444589b10953 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:35.999740 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beab104-1aab-41a4-9836-444589b10953 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:35.999989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9beab104-1aab-41a4-9836-444589b10953 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:36.001600 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 148/296] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:36.006934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15464d83-64ba-4c62-b938-13d949925224 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:36.007485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15464d83-64ba-4c62-b938-13d949925224 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:36.007955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15464d83-64ba-4c62-b938-13d949925224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:36.030566 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 149/297] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:36.035991 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-15464d83-64ba-4c62-b938-13d949925224 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:36.037945 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-15464d83-64ba-4c62-b938-13d949925224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:36.055822 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55b9ee-7322-4e16-b272-6a9290ee6eb7 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:36.129385 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 149/298] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:36.143097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:36.152243 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15464d83-64ba-4c62-b938-13d949925224 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:36.178334 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b55b9ee-7322-4e16-b272-6a9290ee6eb7 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:36.216943 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140cf1e-bd13-47b9-9d9e-7777dd94cd50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:36.218008 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140cf1e-bd13-47b9-9d9e-7777dd94cd50 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:36.218375 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140cf1e-bd13-47b9-9d9e-7777dd94cd50 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:36.218786 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140cf1e-bd13-47b9-9d9e-7777dd94cd50 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:36.230500 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140cf1e-bd13-47b9-9d9e-7777dd94cd50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:36.238431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-9140cf1e-bd13-47b9-9d9e-7777dd94cd50 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 16:53:36.238431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:36.238431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:36.238431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:36.238431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:36.238431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:36.238431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:36.238431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:36.238431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:36.238431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:36.238431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:36.238431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:36.238431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:36.238431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:36.238431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:36.238431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:36.241273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:36.241273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:36.241273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:36.241273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:36.241273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:36.241273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:36.241273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:36.241273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:36.241273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:36.241273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:36.241273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:36.241273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:36.241273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:36.241273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:36.241273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:36.241273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:36.242253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:36.242253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:36.242253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:36.242253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:36.242253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:36.242253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:36.242253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:36.242253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:36.242253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:36.242253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:36.242253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 16:53:36.242253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:36.242253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 150/299] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:53:36.248806 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140cf1e-bd13-47b9-9d9e-7777dd94cd50 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:36.270278 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15464d83-64ba-4c62-b938-13d949925224 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:36.273804 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:36.314956 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f427a-7900-4df2-991b-dd7b97e540ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:36.316570 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f427a-7900-4df2-991b-dd7b97e540ac None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:36.317305 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f427a-7900-4df2-991b-dd7b97e540ac None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:36.317762 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f427a-7900-4df2-991b-dd7b97e540ac None admin] PATH_INFO: `/v3/domains/default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:36.319364 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2f427a-7900-4df2-991b-dd7b97e540ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:36.321318 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2f427a-7900-4df2-991b-dd7b97e540ac None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:36.324261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 150/300] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:53:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:36.330986 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f427a-7900-4df2-991b-dd7b97e540ac None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:36.370949 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140cf1e-bd13-47b9-9d9e-7777dd94cd50 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:36.401779 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:36.408476 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82029448-7b64-499f-b6a8-a69c0fa31ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:36.409143 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82029448-7b64-499f-b6a8-a69c0fa31ace None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:36.409419 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82029448-7b64-499f-b6a8-a69c0fa31ace None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:36.409735 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82029448-7b64-499f-b6a8-a69c0fa31ace None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:36.410482 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82029448-7b64-499f-b6a8-a69c0fa31ace None admin] RBAC: Adding query filter params (name=reader) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:36.410786 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82029448-7b64-499f-b6a8-a69c0fa31ace None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:36.412545 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82029448-7b64-499f-b6a8-a69c0fa31ace None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:36.423013 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 151/301] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:36.427598 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82029448-7b64-499f-b6a8-a69c0fa31ace None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:36.446972 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f427a-7900-4df2-991b-dd7b97e540ac None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:36.483151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0df9eb-497b-441f-a0d8-2d194800e9f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:36.484096 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0df9eb-497b-441f-a0d8-2d194800e9f8 None admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:36.484575 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0df9eb-497b-441f-a0d8-2d194800e9f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:36.485041 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0df9eb-497b-441f-a0d8-2d194800e9f8 None admin] PATH_INFO: `/v3/groups` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:36.485983 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0df9eb-497b-441f-a0d8-2d194800e9f8 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:36.487700 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0df9eb-497b-441f-a0d8-2d194800e9f8 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:36.515488 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 151/302] 10.222.0.13 () {60 vars in 1134 bytes} [Mon May 23 16:53:36 2022] POST /identity/v3/groups => generated 229 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 16:53:36.516966 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629b32a-6b4e-4ffa-9b25-5486f15614fa None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:36.517613 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629b32a-6b4e-4ffa-9b25-5486f15614fa None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:36.517718 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629b32a-6b4e-4ffa-9b25-5486f15614fa None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:36.519358 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 152/303] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:53:36.527063 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7059860-307c-4c3f-8564-a377a146cdc6 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:36.527400 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7059860-307c-4c3f-8564-a377a146cdc6 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:36.527675 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7059860-307c-4c3f-8564-a377a146cdc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:36.547235 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82029448-7b64-499f-b6a8-a69c0fa31ace None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:36.548613 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-c7059860-307c-4c3f-8564-a377a146cdc6 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:36.551200 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-c7059860-307c-4c3f-8564-a377a146cdc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:36.562431 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:36.586696 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc0015-5212-46c9-82a1-64ec5ef021d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:36.587769 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc0015-5212-46c9-82a1-64ec5ef021d9 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:36.588139 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc0015-5212-46c9-82a1-64ec5ef021d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:36.588489 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecc0015-5212-46c9-82a1-64ec5ef021d9 None admin] PATH_INFO: `/v3/users/ee2adb9956f448f38342329d9a1ef2ec` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:36.604423 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc0015-5212-46c9-82a1-64ec5ef021d9 None admin] RBAC: Authorizing `identity:get_user(user_id=ee2adb9956f448f38342329d9a1ef2ec)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:36.606111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecc0015-5212-46c9-82a1-64ec5ef021d9 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:36.608363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 152/304] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:36 2022] GET /identity/v3/users/ee2adb9956f448f38342329d9a1ef2ec => generated 293 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:36.614498 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc0015-5212-46c9-82a1-64ec5ef021d9 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:36.652053 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 153/305] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:36.672493 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7059860-307c-4c3f-8564-a377a146cdc6 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:36.677788 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:36.733127 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecc0015-5212-46c9-82a1-64ec5ef021d9 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:36.770014 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0bc09-3d54-4606-bb01-8d71a65c11d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:36.770659 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0bc09-3d54-4606-bb01-8d71a65c11d4 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:36.770972 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0bc09-3d54-4606-bb01-8d71a65c11d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:36.771281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c0bc09-3d54-4606-bb01-8d71a65c11d4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:36.772114 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0bc09-3d54-4606-bb01-8d71a65c11d4 None admin] RBAC: Adding query filter params (role.id=9d4694dd70b94d68acc78c53ca498bbf, user.id=ee2adb9956f448f38342329d9a1ef2ec, scope.system=all) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:36.772427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0bc09-3d54-4606-bb01-8d71a65c11d4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:36.774721 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96c0bc09-3d54-4606-bb01-8d71a65c11d4 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:36.783007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 153/306] 10.222.0.13 () {56 vars in 1308 bytes} [Mon May 23 16:53:36 2022] GET /identity/v3/role_assignments?user.id=ee2adb9956f448f38342329d9a1ef2ec&scope.system=all&role.id=9d4694dd70b94d68acc78c53ca498bbf => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:36.808816 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7059860-307c-4c3f-8564-a377a146cdc6 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:36.850979 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b46317-494b-450e-bd94-cd5ef4b7790e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:36.852176 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b46317-494b-450e-bd94-cd5ef4b7790e None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:36.852629 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b46317-494b-450e-bd94-cd5ef4b7790e None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:36.853065 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b46317-494b-450e-bd94-cd5ef4b7790e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:36.862028 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:36.867673 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b46317-494b-450e-bd94-cd5ef4b7790e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:36.875736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-e4b46317-494b-450e-bd94-cd5ef4b7790e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:36.875736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:36.875736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:36.875736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:36.875736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:36.875736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:36.875736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:36.875736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:36.875736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:36.875736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:36.875736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:36.875736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:36.875736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:36.875736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:36.875736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:36.875736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:36.876697 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:36.876697 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:36.876697 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:36.876697 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:36.876697 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:36.876697 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:36.876697 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:36.876697 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:36.876697 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:36.876697 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:36.876697 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:36.876697 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:36.876697 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:36.876697 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:36.876697 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:36.876697 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:36.877482 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:36.877482 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:36.877482 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:36.877482 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:36.877482 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:36.877482 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:36.877482 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:36.877482 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:36.877482 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:36.877482 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:36.877482 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:36.877482 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:36.877482 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 154/307] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:53:36.883656 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0bc09-3d54-4606-bb01-8d71a65c11d4 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:36.999170 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c0bc09-3d54-4606-bb01-8d71a65c11d4 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:37.018666 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:37.043185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe0bd7-1fe9-42e0-9da3-5d1588a71c51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:37.043885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe0bd7-1fe9-42e0-9da3-5d1588a71c51 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:37.044163 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe0bd7-1fe9-42e0-9da3-5d1588a71c51 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:37.044433 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-defe0bd7-1fe9-42e0-9da3-5d1588a71c51 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:37.045203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe0bd7-1fe9-42e0-9da3-5d1588a71c51 None admin] RBAC: Adding query filter params (name=member) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:37.045506 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe0bd7-1fe9-42e0-9da3-5d1588a71c51 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:37.046967 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-defe0bd7-1fe9-42e0-9da3-5d1588a71c51 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:37.058258 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 154/308] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:37.067062 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b46317-494b-450e-bd94-cd5ef4b7790e None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:37.183734 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b46317-494b-450e-bd94-cd5ef4b7790e None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:37.221342 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81601e-5cac-4eda-90a4-310fcf89b8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:37.222560 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81601e-5cac-4eda-90a4-310fcf89b8c0 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:37.222842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81601e-5cac-4eda-90a4-310fcf89b8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:37.223150 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81601e-5cac-4eda-90a4-310fcf89b8c0 None admin] PATH_INFO: `/v3/users/c1223f9cfdbf4ae0a971ef90f629493c` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:37.225241 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81601e-5cac-4eda-90a4-310fcf89b8c0 None admin] RBAC: Authorizing `identity:get_user(user_id=c1223f9cfdbf4ae0a971ef90f629493c)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:37.226869 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b81601e-5cac-4eda-90a4-310fcf89b8c0 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:37.229084 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 155/309] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:37 2022] GET /identity/v3/users/c1223f9cfdbf4ae0a971ef90f629493c => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:37.234986 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe0bd7-1fe9-42e0-9da3-5d1588a71c51 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:37.289587 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87527da-d095-43aa-821d-413176954bbb None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:37.289868 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87527da-d095-43aa-821d-413176954bbb None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:37.290068 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87527da-d095-43aa-821d-413176954bbb None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:37.291811 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 156/310] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:53:37.299332 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b85a1-e0e6-4e97-b154-66fb189c69e9 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:37.299857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b85a1-e0e6-4e97-b154-66fb189c69e9 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:37.300337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569b85a1-e0e6-4e97-b154-66fb189c69e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:37.321693 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-569b85a1-e0e6-4e97-b154-66fb189c69e9 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:37.324296 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-569b85a1-e0e6-4e97-b154-66fb189c69e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:37.367420 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-defe0bd7-1fe9-42e0-9da3-5d1588a71c51 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:37.411211 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39272c-8421-4840-88c5-ef84933b77db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:37.412371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39272c-8421-4840-88c5-ef84933b77db None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:37.412811 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39272c-8421-4840-88c5-ef84933b77db None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:37.413086 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd39272c-8421-4840-88c5-ef84933b77db None admin] PATH_INFO: `/v3/system/users/c1223f9cfdbf4ae0a971ef90f629493c/roles/72cc26a1a66c4fc183b39fad4cb38431` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:37.414621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39272c-8421-4840-88c5-ef84933b77db None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=c1223f9cfdbf4ae0a971ef90f629493c, role_id=72cc26a1a66c4fc183b39fad4cb38431)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:37.416658 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd39272c-8421-4840-88c5-ef84933b77db None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:37.419535 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:37.430989 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 155/311] 10.222.0.13 () {58 vars in 1334 bytes} [Mon May 23 16:53:37 2022] PUT /identity/v3/system/users/c1223f9cfdbf4ae0a971ef90f629493c/roles/72cc26a1a66c4fc183b39fad4cb38431 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:53:37.439382 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 157/312] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 16:53:37.484227 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39272c-8421-4840-88c5-ef84933b77db None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:37.544609 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:37.617019 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd39272c-8421-4840-88c5-ef84933b77db None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:37.658282 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02fb937-76ff-4192-a0cf-570fe3db1cbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:37.659476 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02fb937-76ff-4192-a0cf-570fe3db1cbd None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:37.659891 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02fb937-76ff-4192-a0cf-570fe3db1cbd None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:37.660295 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02fb937-76ff-4192-a0cf-570fe3db1cbd None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:37.676622 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02fb937-76ff-4192-a0cf-570fe3db1cbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:37.685147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-e02fb937-76ff-4192-a0cf-570fe3db1cbd None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 16:53:37.685147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:37.685147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:37.685147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:37.685147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:37.685147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:37.685147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:37.685147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:37.685147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:37.685147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:37.685147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:37.685147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:37.685147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:37.685147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:37.685147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:37.685147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:37.686055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:37.686055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:37.686055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:37.686055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:37.686055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:37.686055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:37.686055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:37.686055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:37.686055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:37.686055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:37.686055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:37.686055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:37.686055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:37.686055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:37.686055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:37.686055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:37.686844 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:37.686844 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:37.686844 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:37.686844 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:37.686844 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:37.686844 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:37.686844 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:37.686844 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:37.686844 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:37.686844 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:37.686844 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 16:53:37.686844 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:37.686844 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 156/313] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:37.692332 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b85a1-e0e6-4e97-b154-66fb189c69e9 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:37.797510 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569b85a1-e0e6-4e97-b154-66fb189c69e9 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:37.835555 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107710f-1024-4e16-b01a-97414350a732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:37.836235 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107710f-1024-4e16-b01a-97414350a732 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:37.836728 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107710f-1024-4e16-b01a-97414350a732 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:37.837000 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5107710f-1024-4e16-b01a-97414350a732 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:37.837759 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107710f-1024-4e16-b01a-97414350a732 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:37.838064 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107710f-1024-4e16-b01a-97414350a732 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:37.839508 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5107710f-1024-4e16-b01a-97414350a732 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:37.849149 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 158/314] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:37 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:37.854650 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02fb937-76ff-4192-a0cf-570fe3db1cbd None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:37.966007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02fb937-76ff-4192-a0cf-570fe3db1cbd None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:37.967653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:38.005397 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb3ad2-5a08-477c-929d-a6a5a2505305 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:38.006460 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb3ad2-5a08-477c-929d-a6a5a2505305 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:38.006743 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb3ad2-5a08-477c-929d-a6a5a2505305 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:38.007043 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfb3ad2-5a08-477c-929d-a6a5a2505305 None admin] PATH_INFO: `/v3/users/4c307e6b178a459e806805861b3c11c0` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:38.008255 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfb3ad2-5a08-477c-929d-a6a5a2505305 None admin] RBAC: Authorizing `identity:get_user(user_id=4c307e6b178a459e806805861b3c11c0)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:38.010044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfb3ad2-5a08-477c-929d-a6a5a2505305 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:38.011728 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 157/315] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:37 2022] GET /identity/v3/users/4c307e6b178a459e806805861b3c11c0 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:38.017189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107710f-1024-4e16-b01a-97414350a732 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:38.078482 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:38.122668 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5107710f-1024-4e16-b01a-97414350a732 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:38.168075 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54547311-9e27-4ebb-8d50-33f2f1f45dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:38.169910 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54547311-9e27-4ebb-8d50-33f2f1f45dbd None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:38.170375 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54547311-9e27-4ebb-8d50-33f2f1f45dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:38.170814 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54547311-9e27-4ebb-8d50-33f2f1f45dbd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:38.246310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54547311-9e27-4ebb-8d50-33f2f1f45dbd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:38.248115 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54547311-9e27-4ebb-8d50-33f2f1f45dbd None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:38.308770 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:38.355241 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 159/316] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:53:38.361006 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb3ad2-5a08-477c-929d-a6a5a2505305 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:38.468337 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfb3ad2-5a08-477c-929d-a6a5a2505305 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:38.510138 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fbcd6b-bb2c-4c0d-bbf4-2c75e3858084 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:38.511331 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fbcd6b-bb2c-4c0d-bbf4-2c75e3858084 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:38.511782 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fbcd6b-bb2c-4c0d-bbf4-2c75e3858084 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:38.512216 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fbcd6b-bb2c-4c0d-bbf4-2c75e3858084 None admin] PATH_INFO: `/v3/projects/4206726f0af54455a4cb03012949ffb5` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:38.513548 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fbcd6b-bb2c-4c0d-bbf4-2c75e3858084 None admin] RBAC: Authorizing `identity:get_project(project_id=4206726f0af54455a4cb03012949ffb5)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:38.515209 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fbcd6b-bb2c-4c0d-bbf4-2c75e3858084 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:38.517342 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 158/317] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:38 2022] GET /identity/v3/projects/4206726f0af54455a4cb03012949ffb5 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:38.523489 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54547311-9e27-4ebb-8d50-33f2f1f45dbd None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:38.635027 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54547311-9e27-4ebb-8d50-33f2f1f45dbd None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:38.663565 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9158fab-72a6-436e-b7d7-95670ede325c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:38.664221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9158fab-72a6-436e-b7d7-95670ede325c None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:38.664500 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9158fab-72a6-436e-b7d7-95670ede325c None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:38.664769 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9158fab-72a6-436e-b7d7-95670ede325c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:38.665601 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9158fab-72a6-436e-b7d7-95670ede325c None admin] RBAC: Adding query filter params (role.id=9d4694dd70b94d68acc78c53ca498bbf, scope.project.id=4206726f0af54455a4cb03012949ffb5, user.id=4c307e6b178a459e806805861b3c11c0) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:38.665906 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9158fab-72a6-436e-b7d7-95670ede325c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:38.667376 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9158fab-72a6-436e-b7d7-95670ede325c None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:38.673358 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 160/318] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:53:38 2022] GET /identity/v3/role_assignments?user.id=4c307e6b178a459e806805861b3c11c0&scope.project.id=4206726f0af54455a4cb03012949ffb5&role.id=9d4694dd70b94d68acc78c53ca498bbf => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:38.980695 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb4a24-fc20-4522-adce-363a242b79d4 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:38.981178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb4a24-fc20-4522-adce-363a242b79d4 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:38.981178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfb4a24-fc20-4522-adce-363a242b79d4 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:38.982712 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 159/319] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:38.987728 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e9ac7-fa9e-4393-ac74-aa3b63d0798e None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:38.988212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e9ac7-fa9e-4393-ac74-aa3b63d0798e None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:38.988671 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0e9ac7-fa9e-4393-ac74-aa3b63d0798e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:39.007490 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-8b0e9ac7-fa9e-4393-ac74-aa3b63d0798e None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:39.009348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-8b0e9ac7-fa9e-4393-ac74-aa3b63d0798e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:39.027005 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:39.082953 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 161/320] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:39.101879 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb4a24-fc20-4522-adce-363a242b79d4 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:39.151515 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:39.213677 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecfb4a24-fc20-4522-adce-363a242b79d4 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:39.248141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca00a2d-460f-475d-a2ce-83d0ddb973d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:39.248855 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca00a2d-460f-475d-a2ce-83d0ddb973d8 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:39.249133 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca00a2d-460f-475d-a2ce-83d0ddb973d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:39.249401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca00a2d-460f-475d-a2ce-83d0ddb973d8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:39.250547 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca00a2d-460f-475d-a2ce-83d0ddb973d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:39.252019 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca00a2d-460f-475d-a2ce-83d0ddb973d8 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:39.253639 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 160/321] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:53:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:39.260985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e9ac7-fa9e-4393-ac74-aa3b63d0798e None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:39.297534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc3b05-5d2e-4d5d-a012-ce84048abf7f None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:39.297801 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc3b05-5d2e-4d5d-a012-ce84048abf7f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:39.298247 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbc3b05-5d2e-4d5d-a012-ce84048abf7f None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:39.303136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 161/322] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:39.305295 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da2941-fb1d-43c6-9941-047a61ab7225 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:39.305580 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da2941-fb1d-43c6-9941-047a61ab7225 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:39.305831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2da2941-fb1d-43c6-9941-047a61ab7225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:39.322284 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-e2da2941-fb1d-43c6-9941-047a61ab7225 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:39.323462 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e2da2941-fb1d-43c6-9941-047a61ab7225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:39.374232 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0e9ac7-fa9e-4393-ac74-aa3b63d0798e None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:39.393070 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 162/323] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 16:53:39.415898 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da2941-fb1d-43c6-9941-047a61ab7225 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:39.420333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34e892-4f42-443e-aa4f-5dfb30963f6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:39.421017 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34e892-4f42-443e-aa4f-5dfb30963f6f None admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:39.421295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34e892-4f42-443e-aa4f-5dfb30963f6f None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:39.421564 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34e892-4f42-443e-aa4f-5dfb30963f6f None admin] PATH_INFO: `/v3/groups` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:39.422250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34e892-4f42-443e-aa4f-5dfb30963f6f None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:39.423720 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34e892-4f42-443e-aa4f-5dfb30963f6f None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:39.439828 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 162/324] 10.222.0.13 () {60 vars in 1134 bytes} [Mon May 23 16:53:39 2022] POST /identity/v3/groups => generated 226 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:53:39.456517 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:39.544840 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2da2941-fb1d-43c6-9941-047a61ab7225 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:39.583435 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e56ddd8-edf8-481f-8608-2f324a678135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:39.584742 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56ddd8-edf8-481f-8608-2f324a678135 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:39.585099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56ddd8-edf8-481f-8608-2f324a678135 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:39.585367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56ddd8-edf8-481f-8608-2f324a678135 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:39.597576 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e56ddd8-edf8-481f-8608-2f324a678135 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:39.605464 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-6e56ddd8-edf8-481f-8608-2f324a678135 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 16:53:39.605464 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:39.605464 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:39.605464 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:39.605464 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:39.605464 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:39.605464 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:39.605464 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:39.605464 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:39.605464 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:39.605464 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:39.605464 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:39.605464 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:39.605464 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:39.605464 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:39.605464 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:39.606373 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:39.606373 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:39.606373 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:39.606373 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:39.606373 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:39.606373 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:39.606373 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:39.606373 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:39.606373 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:39.606373 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:39.606373 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:39.606373 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:39.606373 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:39.606373 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:39.606373 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:39.606373 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:39.607642 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:39.607642 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:39.607642 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:39.607642 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:39.607642 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:39.607642 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:39.607642 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:39.607642 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:39.607642 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:39.607642 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:39.607642 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 16:53:39.607642 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:39.607642 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 163/325] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:39.612568 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34e892-4f42-443e-aa4f-5dfb30963f6f None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:39.629343 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:39.711041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55b73e-2968-4c90-8cde-f7e6746d4d52 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:39.711332 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55b73e-2968-4c90-8cde-f7e6746d4d52 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:39.711604 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55b73e-2968-4c90-8cde-f7e6746d4d52 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:39.712995 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 164/326] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:39.718559 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da85d70-c4e9-408c-b54d-1ea1e66e2175 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:39.718842 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da85d70-c4e9-408c-b54d-1ea1e66e2175 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:39.719155 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da85d70-c4e9-408c-b54d-1ea1e66e2175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:39.733542 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34e892-4f42-443e-aa4f-5dfb30963f6f None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:39.742343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-3da85d70-c4e9-408c-b54d-1ea1e66e2175 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:39.743605 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-3da85d70-c4e9-408c-b54d-1ea1e66e2175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:39.780693 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfffd7-4ea9-4809-ab42-55f958c8876a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:39.781382 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfffd7-4ea9-4809-ab42-55f958c8876a None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:39.781666 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfffd7-4ea9-4809-ab42-55f958c8876a None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:39.781940 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfffd7-4ea9-4809-ab42-55f958c8876a None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:39.782703 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfffd7-4ea9-4809-ab42-55f958c8876a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:39.783091 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfffd7-4ea9-4809-ab42-55f958c8876a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:39.784482 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfffd7-4ea9-4809-ab42-55f958c8876a None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:39.796036 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 163/327] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:39.804311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfffd7-4ea9-4809-ab42-55f958c8876a None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:39.842964 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 165/328] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 16:53:39.869365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da85d70-c4e9-408c-b54d-1ea1e66e2175 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:39.902035 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:39.911591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:39.946113 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfffd7-4ea9-4809-ab42-55f958c8876a None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:39.995219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd846b9c-b4e0-43af-b0fe-486d304af249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:39.996209 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd846b9c-b4e0-43af-b0fe-486d304af249 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:39.996555 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd846b9c-b4e0-43af-b0fe-486d304af249 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:39.996890 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd846b9c-b4e0-43af-b0fe-486d304af249 None admin] PATH_INFO: `/v3/users/ee2adb9956f448f38342329d9a1ef2ec` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:39.997865 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da85d70-c4e9-408c-b54d-1ea1e66e2175 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:39.998308 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd846b9c-b4e0-43af-b0fe-486d304af249 None admin] RBAC: Authorizing `identity:get_user(user_id=ee2adb9956f448f38342329d9a1ef2ec)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:39.999765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd846b9c-b4e0-43af-b0fe-486d304af249 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:40.002063 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 164/329] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:39 2022] GET /identity/v3/users/ee2adb9956f448f38342329d9a1ef2ec => generated 293 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:40.008457 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd846b9c-b4e0-43af-b0fe-486d304af249 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:40.037417 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73826236-92c2-49ce-acaa-7b32a8f4d413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:40.038567 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73826236-92c2-49ce-acaa-7b32a8f4d413 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:40.039016 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73826236-92c2-49ce-acaa-7b32a8f4d413 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:40.039425 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73826236-92c2-49ce-acaa-7b32a8f4d413 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:40.050999 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73826236-92c2-49ce-acaa-7b32a8f4d413 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:40.059654 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-73826236-92c2-49ce-acaa-7b32a8f4d413 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:40.059654 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:40.059654 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:40.059654 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:40.059654 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:40.059654 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:40.059654 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:40.059654 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:40.059654 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:40.059654 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:40.059654 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:40.059654 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:40.059654 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:40.059654 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:40.059654 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:40.059654 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:40.060532 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:40.060532 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:40.060532 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:40.060532 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:40.060532 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:40.060532 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:40.060532 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:40.060532 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:40.060532 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:40.060532 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:40.060532 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:40.060532 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:40.060532 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:40.060532 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:40.060532 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:40.060532 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:40.061324 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:40.061324 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:40.061324 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:40.061324 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:40.061324 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:40.061324 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:40.061324 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:40.061324 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:40.061324 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:40.061324 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:40.061324 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:40.061324 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:40.064256 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 166/330] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:40.074187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73826236-92c2-49ce-acaa-7b32a8f4d413 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:40.126054 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd846b9c-b4e0-43af-b0fe-486d304af249 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:40.177938 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4c9b6-a1a0-4fae-9bdf-0ba5348e6c2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:40.178816 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4c9b6-a1a0-4fae-9bdf-0ba5348e6c2a None admin] REQUEST_METHOD: `PUT` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:40.179125 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4c9b6-a1a0-4fae-9bdf-0ba5348e6c2a None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:40.179385 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc4c9b6-a1a0-4fae-9bdf-0ba5348e6c2a None admin] PATH_INFO: `/v3/system/users/ee2adb9956f448f38342329d9a1ef2ec/roles/9d4694dd70b94d68acc78c53ca498bbf` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:40.180576 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:40.181696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4c9b6-a1a0-4fae-9bdf-0ba5348e6c2a None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=ee2adb9956f448f38342329d9a1ef2ec, role_id=9d4694dd70b94d68acc78c53ca498bbf)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:40.184412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc4c9b6-a1a0-4fae-9bdf-0ba5348e6c2a None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:40.200163 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 165/331] 10.222.0.13 () {58 vars in 1334 bytes} [Mon May 23 16:53:40 2022] PUT /identity/v3/system/users/ee2adb9956f448f38342329d9a1ef2ec/roles/9d4694dd70b94d68acc78c53ca498bbf => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 16:53:40.200697 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73826236-92c2-49ce-acaa-7b32a8f4d413 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:40.242781 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afa41a-07f3-4cd5-b631-b6791abbb236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:40.243898 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afa41a-07f3-4cd5-b631-b6791abbb236 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:40.244171 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afa41a-07f3-4cd5-b631-b6791abbb236 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:40.244448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62afa41a-07f3-4cd5-b631-b6791abbb236 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:40.245227 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afa41a-07f3-4cd5-b631-b6791abbb236 None admin] RBAC: Adding query filter params (name=member) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:40.245536 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afa41a-07f3-4cd5-b631-b6791abbb236 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:40.247076 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62afa41a-07f3-4cd5-b631-b6791abbb236 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:40.256883 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 167/332] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:40.262985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4c9b6-a1a0-4fae-9bdf-0ba5348e6c2a None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:40.287335 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:40.382732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc4c9b6-a1a0-4fae-9bdf-0ba5348e6c2a None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:40.420762 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75729055-567f-4a07-8c0f-434d6252976c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:40.421672 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75729055-567f-4a07-8c0f-434d6252976c None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:40.421955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75729055-567f-4a07-8c0f-434d6252976c None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:40.422228 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75729055-567f-4a07-8c0f-434d6252976c None admin] PATH_INFO: `/v3/users/c1223f9cfdbf4ae0a971ef90f629493c` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:40.423540 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75729055-567f-4a07-8c0f-434d6252976c None admin] RBAC: Authorizing `identity:get_user(user_id=c1223f9cfdbf4ae0a971ef90f629493c)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:40.425026 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75729055-567f-4a07-8c0f-434d6252976c None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:40.426795 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 166/333] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:40 2022] GET /identity/v3/users/c1223f9cfdbf4ae0a971ef90f629493c => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:40.432791 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afa41a-07f3-4cd5-b631-b6791abbb236 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:40.569238 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62afa41a-07f3-4cd5-b631-b6791abbb236 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:40.615663 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ec8ecf-89d9-48d4-8dbf-e8bb5eb6c4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:40.616337 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec8ecf-89d9-48d4-8dbf-e8bb5eb6c4de None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:40.616617 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec8ecf-89d9-48d4-8dbf-e8bb5eb6c4de None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:40.616891 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ec8ecf-89d9-48d4-8dbf-e8bb5eb6c4de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:40.617725 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec8ecf-89d9-48d4-8dbf-e8bb5eb6c4de None admin] RBAC: Adding query filter params (role.id=72cc26a1a66c4fc183b39fad4cb38431, user.id=c1223f9cfdbf4ae0a971ef90f629493c, scope.system=all) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:40.618035 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec8ecf-89d9-48d4-8dbf-e8bb5eb6c4de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:40.619565 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ec8ecf-89d9-48d4-8dbf-e8bb5eb6c4de None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:40.637144 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 168/334] 10.222.0.13 () {56 vars in 1308 bytes} [Mon May 23 16:53:40 2022] GET /identity/v3/role_assignments?user.id=c1223f9cfdbf4ae0a971ef90f629493c&scope.system=all&role.id=72cc26a1a66c4fc183b39fad4cb38431 => generated 510 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:42.203708 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf213c-222a-405c-9aaa-b97934e2f208 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:42.204776 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf213c-222a-405c-9aaa-b97934e2f208 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:42.205252 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaf213c-222a-405c-9aaa-b97934e2f208 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:42.207068 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 167/335] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:42.213122 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af2507-c54e-43fb-9772-f785b84133c7 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:42.213989 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af2507-c54e-43fb-9772-f785b84133c7 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:42.214472 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31af2507-c54e-43fb-9772-f785b84133c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:42.227158 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:42.240327 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-31af2507-c54e-43fb-9772-f785b84133c7 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:42.242842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4632b7-f9dc-4773-8ba4-843c21707466 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:42.243152 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4632b7-f9dc-4773-8ba4-843c21707466 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:42.243428 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4632b7-f9dc-4773-8ba4-843c21707466 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:42.244791 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 168/336] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:53:42.245149 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-31af2507-c54e-43fb-9772-f785b84133c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:42.250520 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e42ee2-e251-48a3-9c46-2094074ceab5 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:42.250803 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e42ee2-e251-48a3-9c46-2094074ceab5 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:42.251104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e42ee2-e251-48a3-9c46-2094074ceab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:42.278153 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-b5e42ee2-e251-48a3-9c46-2094074ceab5 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:42.279362 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-b5e42ee2-e251-48a3-9c46-2094074ceab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:42.388156 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 169/337] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 16:53:42.413338 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 169/338] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 16:53:42.426353 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31af2507-c54e-43fb-9772-f785b84133c7 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:42.441572 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e42ee2-e251-48a3-9c46-2094074ceab5 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:42.445585 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:42.588361 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31af2507-c54e-43fb-9772-f785b84133c7 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:42.595062 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e42ee2-e251-48a3-9c46-2094074ceab5 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:42.630416 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84884772-b3ae-46ee-af21-f18b8e8c00dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:42.632051 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84884772-b3ae-46ee-af21-f18b8e8c00dd None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:42.632051 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84884772-b3ae-46ee-af21-f18b8e8c00dd None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:42.632051 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84884772-b3ae-46ee-af21-f18b8e8c00dd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:42.639791 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202c593-ee96-4e5d-a250-28ca68ae308e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:42.641131 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202c593-ee96-4e5d-a250-28ca68ae308e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:42.641629 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202c593-ee96-4e5d-a250-28ca68ae308e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:42.642099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5202c593-ee96-4e5d-a250-28ca68ae308e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:42.647529 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:42.648566 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84884772-b3ae-46ee-af21-f18b8e8c00dd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:42.658090 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-84884772-b3ae-46ee-af21-f18b8e8c00dd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:42.658090 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:42.658090 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:42.658090 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:42.658090 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:42.658090 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:42.658090 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:42.658090 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:42.658090 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:42.658090 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:42.658090 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:42.658090 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:42.658090 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:42.658090 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:42.658090 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:42.658090 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:42.659168 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:42.659168 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:42.659168 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:42.659168 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:42.659168 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:42.659168 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:42.659168 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:42.659168 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:42.659168 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:42.659168 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:42.659168 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:42.659168 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:42.659168 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:42.659168 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:42.659168 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:42.659168 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:42.659962 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:42.659962 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:42.659962 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:42.659962 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:42.659962 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:42.659962 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:42.659962 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:42.659962 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:42.659962 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:42.659962 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:42.659962 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:42.659962 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:42.659962 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 170/339] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:42 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:53:42.667494 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84884772-b3ae-46ee-af21-f18b8e8c00dd None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:42.668799 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5202c593-ee96-4e5d-a250-28ca68ae308e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:42.681187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-5202c593-ee96-4e5d-a250-28ca68ae308e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:53:42.681187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:42.681187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:42.681187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:42.681187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:42.681187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:42.681187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:42.681187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:42.681187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:42.681187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:42.681187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:42.681187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:42.681187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:42.681187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:42.681187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:42.681187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:42.682119 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:42.682119 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:42.682119 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:42.682119 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:42.682119 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:42.682119 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:42.682119 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:42.682119 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:42.682119 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:42.682119 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:42.682119 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:42.682119 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:42.682119 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:42.682119 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:42.682119 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:42.682119 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:42.682948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:42.682948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:42.682948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:42.682948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:42.682948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:42.682948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:42.682948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:42.682948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:42.682948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:42.682948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:42.682948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:53:42.682948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:42.685682 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 170/340] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 23 16:53:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:42.708918 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202c593-ee96-4e5d-a250-28ca68ae308e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:42.751459 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:42.790534 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84884772-b3ae-46ee-af21-f18b8e8c00dd None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:42.829877 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e050a-f69a-441b-9463-2d4757ac046c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:42.830564 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e050a-f69a-441b-9463-2d4757ac046c None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:42.830848 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e050a-f69a-441b-9463-2d4757ac046c None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:42.831193 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e050a-f69a-441b-9463-2d4757ac046c None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:42.832058 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e050a-f69a-441b-9463-2d4757ac046c None admin] RBAC: Adding query filter params (name=member) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:42.832373 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e050a-f69a-441b-9463-2d4757ac046c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:42.834178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e050a-f69a-441b-9463-2d4757ac046c None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:42.843653 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 171/341] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:42.864908 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5202c593-ee96-4e5d-a250-28ca68ae308e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:42.867863 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e050a-f69a-441b-9463-2d4757ac046c None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:42.911684 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af54c7-2f3b-41c4-8143-6a330f85a843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:42.912379 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af54c7-2f3b-41c4-8143-6a330f85a843 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:42.912661 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af54c7-2f3b-41c4-8143-6a330f85a843 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:42.912934 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65af54c7-2f3b-41c4-8143-6a330f85a843 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:42.913698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af54c7-2f3b-41c4-8143-6a330f85a843 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:42.914007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af54c7-2f3b-41c4-8143-6a330f85a843 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:42.915467 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65af54c7-2f3b-41c4-8143-6a330f85a843 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:42.924518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 171/342] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:53:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:42.937112 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af54c7-2f3b-41c4-8143-6a330f85a843 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:43.037224 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e050a-f69a-441b-9463-2d4757ac046c None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:43.059288 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:43.088415 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:43.106400 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f9074-1e90-4788-99b3-4398aaca8e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:43.107162 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f9074-1e90-4788-99b3-4398aaca8e4b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:43.107445 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f9074-1e90-4788-99b3-4398aaca8e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:43.107718 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6f9074-1e90-4788-99b3-4398aaca8e4b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:43.136173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65af54c7-2f3b-41c4-8143-6a330f85a843 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:43.207534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e867d0-1e95-42c1-8c30-dc7b517d065e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:43.208703 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e867d0-1e95-42c1-8c30-dc7b517d065e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:43.209227 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e867d0-1e95-42c1-8c30-dc7b517d065e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:43.209695 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e867d0-1e95-42c1-8c30-dc7b517d065e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:43.213914 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f9074-1e90-4788-99b3-4398aaca8e4b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:43.215485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6f9074-1e90-4788-99b3-4398aaca8e4b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:43.358257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 172/343] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 16:53:43.365433 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f9074-1e90-4788-99b3-4398aaca8e4b None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:43.387365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e867d0-1e95-42c1-8c30-dc7b517d065e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:43.389533 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e867d0-1e95-42c1-8c30-dc7b517d065e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:43.413347 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:43.501907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:43.535330 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6f9074-1e90-4788-99b3-4398aaca8e4b None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:43.554287 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 172/344] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 617 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:53:43.555686 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5bd90-1408-4ca7-ad73-f18c9e339316 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:43.555981 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5bd90-1408-4ca7-ad73-f18c9e339316 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:43.556257 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d5bd90-1408-4ca7-ad73-f18c9e339316 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:43.557693 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 173/345] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:43.558595 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e13a9-3296-4465-b336-0327acf41c83 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:43.558874 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e13a9-3296-4465-b336-0327acf41c83 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:43.559182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38e13a9-3296-4465-b336-0327acf41c83 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:43.560578 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 174/346] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:43.564526 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e13a9-3296-4465-b336-0327acf41c83 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:43.576540 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535e685-eefe-4191-8295-5e0f52054704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:43.577419 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535e685-eefe-4191-8295-5e0f52054704 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:43.577697 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535e685-eefe-4191-8295-5e0f52054704 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:43.577966 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535e685-eefe-4191-8295-5e0f52054704 None admin] PATH_INFO: `/v3/projects/f29295d0fb6141208e3a094c3be29944` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:43.579188 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535e685-eefe-4191-8295-5e0f52054704 None admin] RBAC: Authorizing `identity:get_project(project_id=f29295d0fb6141208e3a094c3be29944)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:43.580645 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535e685-eefe-4191-8295-5e0f52054704 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:43.582614 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 173/347] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:43 2022] GET /identity/v3/projects/f29295d0fb6141208e3a094c3be29944 => generated 298 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:43.584053 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c890e7b-721b-4357-9c31-b3cd474f155a None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:43.584344 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c890e7b-721b-4357-9c31-b3cd474f155a None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:43.584613 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c890e7b-721b-4357-9c31-b3cd474f155a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:43.611802 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-7c890e7b-721b-4357-9c31-b3cd474f155a None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:43.613559 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-7c890e7b-721b-4357-9c31-b3cd474f155a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:43.697745 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38e13a9-3296-4465-b336-0327acf41c83 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:43.700335 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 174/348] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:43.702672 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e31af9-5cf6-446a-aa64-d4db5dabc7eb None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:43.703075 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e31af9-5cf6-446a-aa64-d4db5dabc7eb None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:43.703421 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e31af9-5cf6-446a-aa64-d4db5dabc7eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:43.723972 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-62e31af9-5cf6-446a-aa64-d4db5dabc7eb None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:43.725653 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-62e31af9-5cf6-446a-aa64-d4db5dabc7eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:43.740289 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce3afa-dafd-4ff2-8e59-f265a69b168b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:43.741321 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce3afa-dafd-4ff2-8e59-f265a69b168b None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:43.741606 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce3afa-dafd-4ff2-8e59-f265a69b168b None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:43.741880 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ce3afa-dafd-4ff2-8e59-f265a69b168b None admin] PATH_INFO: `/v3/projects/f29295d0fb6141208e3a094c3be29944` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:43.743317 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce3afa-dafd-4ff2-8e59-f265a69b168b None admin] RBAC: Authorizing `identity:get_project(project_id=f29295d0fb6141208e3a094c3be29944)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:43.744883 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48ce3afa-dafd-4ff2-8e59-f265a69b168b None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:43.746867 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 175/349] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:43 2022] GET /identity/v3/projects/f29295d0fb6141208e3a094c3be29944 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:43.747457 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:43.748721 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce3afa-dafd-4ff2-8e59-f265a69b168b None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:43.787556 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:43.824123 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 175/350] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:43.825342 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e31af9-5cf6-446a-aa64-d4db5dabc7eb None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:43.876166 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ce3afa-dafd-4ff2-8e59-f265a69b168b None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:43.914993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2508da-21b9-4049-8e73-cd7c57ebd3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:43.916290 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2508da-21b9-4049-8e73-cd7c57ebd3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:43.916760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2508da-21b9-4049-8e73-cd7c57ebd3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:43.917216 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2508da-21b9-4049-8e73-cd7c57ebd3d9 None admin] PATH_INFO: `/v3/groups/df89dd397b0e496fad775ebf220818d2` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:43.924024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2508da-21b9-4049-8e73-cd7c57ebd3d9 None admin] RBAC: Authorizing `identity:get_group(group_id=df89dd397b0e496fad775ebf220818d2)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:43.925775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2508da-21b9-4049-8e73-cd7c57ebd3d9 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:43.928327 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 176/351] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 23 16:53:43 2022] GET /identity/v3/groups/df89dd397b0e496fad775ebf220818d2 => generated 226 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:43.929936 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2508da-21b9-4049-8e73-cd7c57ebd3d9 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:43.943920 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e31af9-5cf6-446a-aa64-d4db5dabc7eb None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:43.985771 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26206f-f8ec-46a7-877e-414e98ddca6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:43.986593 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26206f-f8ec-46a7-877e-414e98ddca6f None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:43.987127 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26206f-f8ec-46a7-877e-414e98ddca6f None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:43.987299 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d26206f-f8ec-46a7-877e-414e98ddca6f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:43.999797 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d26206f-f8ec-46a7-877e-414e98ddca6f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:44.008151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-3d26206f-f8ec-46a7-877e-414e98ddca6f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:44.008151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:44.008151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:44.008151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:44.008151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:44.008151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:44.008151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:44.008151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:44.008151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:44.008151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:44.008151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:44.008151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:44.008151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:44.008151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:44.008151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:44.008151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:44.009475 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:44.009475 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:44.009475 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:44.009475 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:44.009475 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:44.009475 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:44.009475 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:44.009475 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:44.009475 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:44.009475 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:44.009475 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:44.009475 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:44.009475 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:44.009475 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:44.009475 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:44.009475 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:44.015277 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:44.015277 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:44.015277 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:44.015277 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:44.015277 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:44.015277 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:44.015277 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:44.015277 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:44.015277 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:44.015277 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:44.015277 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:44.015277 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:44.015277 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 176/352] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:44.015277 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26206f-f8ec-46a7-877e-414e98ddca6f None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:44.033618 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:44.041792 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:44.053155 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2508da-21b9-4049-8e73-cd7c57ebd3d9 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:44.091225 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79e693-4dc0-42c2-9f0c-f8496d02ab24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:44.092434 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79e693-4dc0-42c2-9f0c-f8496d02ab24 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:44.092902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79e693-4dc0-42c2-9f0c-f8496d02ab24 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:44.093387 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f79e693-4dc0-42c2-9f0c-f8496d02ab24 None admin] PATH_INFO: `/v3/groups/df89dd397b0e496fad775ebf220818d2` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:44.094731 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79e693-4dc0-42c2-9f0c-f8496d02ab24 None admin] RBAC: Authorizing `identity:get_group(group_id=df89dd397b0e496fad775ebf220818d2)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:44.096420 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f79e693-4dc0-42c2-9f0c-f8496d02ab24 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:44.098456 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 177/353] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 23 16:53:43 2022] GET /identity/v3/groups/df89dd397b0e496fad775ebf220818d2 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:44.099982 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79e693-4dc0-42c2-9f0c-f8496d02ab24 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:44.130477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d26206f-f8ec-46a7-877e-414e98ddca6f None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:44.167732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0e1c8-3952-4236-88f2-0666053f6bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:44.168541 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0e1c8-3952-4236-88f2-0666053f6bad None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:44.169008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0e1c8-3952-4236-88f2-0666053f6bad None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:44.169110 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa0e1c8-3952-4236-88f2-0666053f6bad None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:44.181504 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa0e1c8-3952-4236-88f2-0666053f6bad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:44.189385 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-daa0e1c8-3952-4236-88f2-0666053f6bad None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 16:53:44.189385 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:44.189385 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:44.189385 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:44.189385 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:44.189385 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:44.189385 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:44.189385 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:44.189385 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:44.189385 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:44.189385 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:44.189385 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:44.189385 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:44.189385 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:44.189385 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:44.189385 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:44.190291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:44.190291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:44.190291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:44.190291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:44.190291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:44.190291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:44.190291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:44.190291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:44.190291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:44.190291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:44.190291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:44.190291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:44.190291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:44.190291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:44.190291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:44.190291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:44.191479 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:44.191479 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:44.191479 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:44.191479 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:44.191479 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:44.191479 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:44.191479 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:44.191479 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:44.191479 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:44.191479 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:44.191479 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 16:53:44.191479 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:44.191479 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 177/354] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:44 2022] GET /identity/v3/roles/reader => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:44.192127 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0e1c8-3952-4236-88f2-0666053f6bad None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:44.213179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f79e693-4dc0-42c2-9f0c-f8496d02ab24 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:44.251476 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eaacc3-2990-4e9c-968a-3c0d77f8d716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:44.254270 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eaacc3-2990-4e9c-968a-3c0d77f8d716 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:44.254270 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eaacc3-2990-4e9c-968a-3c0d77f8d716 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:44.254270 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eaacc3-2990-4e9c-968a-3c0d77f8d716 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:44.254270 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eaacc3-2990-4e9c-968a-3c0d77f8d716 None admin] RBAC: Adding query filter params (group.id=df89dd397b0e496fad775ebf220818d2, role.id=72cc26a1a66c4fc183b39fad4cb38431, scope.project.id=f29295d0fb6141208e3a094c3be29944) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:44.254270 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eaacc3-2990-4e9c-968a-3c0d77f8d716 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:44.257704 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eaacc3-2990-4e9c-968a-3c0d77f8d716 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:44.263472 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 178/355] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 23 16:53:44 2022] GET /identity/v3/role_assignments?group.id=df89dd397b0e496fad775ebf220818d2&scope.project.id=f29295d0fb6141208e3a094c3be29944&role.id=72cc26a1a66c4fc183b39fad4cb38431 => generated 260 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:44.264601 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eaacc3-2990-4e9c-968a-3c0d77f8d716 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:44.298862 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:44.307980 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:44.339181 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa0e1c8-3952-4236-88f2-0666053f6bad None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:44.385341 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef1b919-4133-4af1-80d2-38228b07e0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:44.386323 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1b919-4133-4af1-80d2-38228b07e0ec None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:44.386848 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1b919-4133-4af1-80d2-38228b07e0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:44.387312 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef1b919-4133-4af1-80d2-38228b07e0ec None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:44.388434 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1b919-4133-4af1-80d2-38228b07e0ec None admin] RBAC: Adding query filter params (name=member) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:44.388965 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1b919-4133-4af1-80d2-38228b07e0ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:44.390656 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef1b919-4133-4af1-80d2-38228b07e0ec None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:44.404259 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 178/356] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:44.406381 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818afea4-70a8-49b0-941a-e8a13c8584b1 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:44.406877 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818afea4-70a8-49b0-941a-e8a13c8584b1 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:44.407387 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818afea4-70a8-49b0-941a-e8a13c8584b1 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:44.408692 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eaacc3-2990-4e9c-968a-3c0d77f8d716 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:44.409557 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 179/357] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:44.410546 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818afea4-70a8-49b0-941a-e8a13c8584b1 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:44.447936 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914cba3-d632-4929-bc80-47ada6512ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:44.448703 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914cba3-d632-4929-bc80-47ada6512ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:44.449154 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914cba3-d632-4929-bc80-47ada6512ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:44.449345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2914cba3-d632-4929-bc80-47ada6512ac6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:44.450291 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914cba3-d632-4929-bc80-47ada6512ac6 None admin] RBAC: Adding query filter params (group.id=df89dd397b0e496fad775ebf220818d2, role.id=6c30aca17fad4c4fa40b9461c5bda291, scope.project.id=f29295d0fb6141208e3a094c3be29944) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:44.450675 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914cba3-d632-4929-bc80-47ada6512ac6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:44.452255 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2914cba3-d632-4929-bc80-47ada6512ac6 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:44.458712 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 179/358] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 23 16:53:44 2022] GET /identity/v3/role_assignments?group.id=df89dd397b0e496fad775ebf220818d2&scope.project.id=f29295d0fb6141208e3a094c3be29944&role.id=6c30aca17fad4c4fa40b9461c5bda291 => generated 260 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:44.459816 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914cba3-d632-4929-bc80-47ada6512ac6 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:44.541742 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818afea4-70a8-49b0-941a-e8a13c8584b1 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:44.584142 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569cf9f1-6bba-49ef-a97f-94852d0e0132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:44.585204 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cf9f1-6bba-49ef-a97f-94852d0e0132 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:44.585674 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cf9f1-6bba-49ef-a97f-94852d0e0132 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:44.586133 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569cf9f1-6bba-49ef-a97f-94852d0e0132 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:44.587413 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569cf9f1-6bba-49ef-a97f-94852d0e0132 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:44.587946 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569cf9f1-6bba-49ef-a97f-94852d0e0132 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:44.590266 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569cf9f1-6bba-49ef-a97f-94852d0e0132 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:44.601191 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 180/359] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:44.601615 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:44.609224 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092c4c9-29fb-4d69-bab7-10ec93e7cec8 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:44.609857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092c4c9-29fb-4d69-bab7-10ec93e7cec8 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:44.610328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4092c4c9-29fb-4d69-bab7-10ec93e7cec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:44.630744 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-4092c4c9-29fb-4d69-bab7-10ec93e7cec8 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:44.632580 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-4092c4c9-29fb-4d69-bab7-10ec93e7cec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:44.711922 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 181/360] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:44.713373 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092c4c9-29fb-4d69-bab7-10ec93e7cec8 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:44.720328 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:44.772620 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2914cba3-d632-4929-bc80-47ada6512ac6 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:44.829521 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb18b0e-93b3-4237-9ae8-0756d2a1da5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:44.830294 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb18b0e-93b3-4237-9ae8-0756d2a1da5f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:44.830640 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb18b0e-93b3-4237-9ae8-0756d2a1da5f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:44.831003 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb18b0e-93b3-4237-9ae8-0756d2a1da5f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:44.842816 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4092c4c9-29fb-4d69-bab7-10ec93e7cec8 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:44.882481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf0bafd-ba50-4123-a7fd-342e2a8e0f48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:44.883721 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0bafd-ba50-4123-a7fd-342e2a8e0f48 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:44.884598 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0bafd-ba50-4123-a7fd-342e2a8e0f48 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:44.885187 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf0bafd-ba50-4123-a7fd-342e2a8e0f48 None admin] PATH_INFO: `/v3/users/ee2adb9956f448f38342329d9a1ef2ec` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:44.886880 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf0bafd-ba50-4123-a7fd-342e2a8e0f48 None admin] RBAC: Authorizing `identity:get_user(user_id=ee2adb9956f448f38342329d9a1ef2ec)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:44.889067 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaf0bafd-ba50-4123-a7fd-342e2a8e0f48 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:44.891554 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 182/361] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:53:44 2022] GET /identity/v3/users/ee2adb9956f448f38342329d9a1ef2ec => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:44.896175 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:44.897099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf0bafd-ba50-4123-a7fd-342e2a8e0f48 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:45.006870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf0bafd-ba50-4123-a7fd-342e2a8e0f48 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:45.026377 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb18b0e-93b3-4237-9ae8-0756d2a1da5f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:45.027899 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb18b0e-93b3-4237-9ae8-0756d2a1da5f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:45.051685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734e68b-b127-489f-acd8-53ae817a7e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:45.052954 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734e68b-b127-489f-acd8-53ae817a7e97 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:45.053407 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734e68b-b127-489f-acd8-53ae817a7e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:45.054376 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4734e68b-b127-489f-acd8-53ae817a7e97 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:45.069600 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4734e68b-b127-489f-acd8-53ae817a7e97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:45.079870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-4734e68b-b127-489f-acd8-53ae817a7e97 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:53:45.079870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:45.079870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:45.079870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:45.079870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:45.079870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:45.079870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:45.079870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:45.079870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:45.079870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:45.079870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:45.079870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:45.079870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:45.079870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:45.079870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:45.079870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:45.080906 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:45.080906 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:45.080906 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:45.080906 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:45.080906 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:45.080906 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:45.080906 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:45.080906 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:45.080906 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:45.080906 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:45.080906 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:45.080906 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:45.080906 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:45.080906 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:45.080906 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:45.080906 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:45.081707 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:45.081707 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:45.081707 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:45.081707 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:45.081707 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:45.081707 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:45.081707 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:45.081707 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:45.081707 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:45.081707 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:45.081707 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:53:45.081707 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:45.084875 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 183/362] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 23 16:53:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:45.086169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734e68b-b127-489f-acd8-53ae817a7e97 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:45.132853 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:45.153326 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 180/363] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 694 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:53:45.154485 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57392089-a04f-4621-a178-1981929948a7 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:45.154773 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57392089-a04f-4621-a178-1981929948a7 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:45.159195 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57392089-a04f-4621-a178-1981929948a7 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:45.160747 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 181/364] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:45.161599 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57392089-a04f-4621-a178-1981929948a7 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:45.163353 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:45.225941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4734e68b-b127-489f-acd8-53ae817a7e97 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:45.275476 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44002cae-e083-4610-8d25-2c00b6b1b0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:45.276613 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44002cae-e083-4610-8d25-2c00b6b1b0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:45.276994 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44002cae-e083-4610-8d25-2c00b6b1b0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:45.277368 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44002cae-e083-4610-8d25-2c00b6b1b0b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:45.278313 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44002cae-e083-4610-8d25-2c00b6b1b0b3 None admin] RBAC: Adding query filter params (role.id=9d4694dd70b94d68acc78c53ca498bbf, user.id=ee2adb9956f448f38342329d9a1ef2ec, scope.system=all) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:45.279253 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44002cae-e083-4610-8d25-2c00b6b1b0b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:45.280956 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44002cae-e083-4610-8d25-2c00b6b1b0b3 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:45.296598 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57392089-a04f-4621-a178-1981929948a7 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:45.300781 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 184/365] 10.222.0.13 () {56 vars in 1308 bytes} [Mon May 23 16:53:45 2022] GET /identity/v3/role_assignments?user.id=ee2adb9956f448f38342329d9a1ef2ec&scope.system=all&role.id=9d4694dd70b94d68acc78c53ca498bbf => generated 510 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:45.302361 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf28d7-e469-494d-b274-e3aa4e336bde None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:45.302760 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf28d7-e469-494d-b274-e3aa4e336bde None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:45.303362 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaf28d7-e469-494d-b274-e3aa4e336bde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:45.326778 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-5aaf28d7-e469-494d-b274-e3aa4e336bde None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:45.328835 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-5aaf28d7-e469-494d-b274-e3aa4e336bde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:45.345568 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e058222-7a8e-446c-a7fd-71df52729fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:45.346263 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e058222-7a8e-446c-a7fd-71df52729fb5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:45.346588 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e058222-7a8e-446c-a7fd-71df52729fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:45.346865 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e058222-7a8e-446c-a7fd-71df52729fb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:45.347687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e058222-7a8e-446c-a7fd-71df52729fb5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:45.347999 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e058222-7a8e-446c-a7fd-71df52729fb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:45.352262 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e058222-7a8e-446c-a7fd-71df52729fb5 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:45.365662 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 182/366] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:53:45 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:45.366374 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e058222-7a8e-446c-a7fd-71df52729fb5 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:45.432618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:45.491046 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e058222-7a8e-446c-a7fd-71df52729fb5 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:45.531577 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1aa161-69a6-430e-9dd1-111120e51048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:45.532384 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1aa161-69a6-430e-9dd1-111120e51048 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:45.532666 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1aa161-69a6-430e-9dd1-111120e51048 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:45.532936 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1aa161-69a6-430e-9dd1-111120e51048 None admin] PATH_INFO: `/v3/projects/4206726f0af54455a4cb03012949ffb5` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:45.533890 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 185/367] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:45.534448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1aa161-69a6-430e-9dd1-111120e51048 None admin] RBAC: Authorizing `identity:get_project(project_id=4206726f0af54455a4cb03012949ffb5)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:45.536530 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf28d7-e469-494d-b274-e3aa4e336bde None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:45.540626 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1aa161-69a6-430e-9dd1-111120e51048 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:45.542793 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 183/368] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:45 2022] GET /identity/v3/projects/4206726f0af54455a4cb03012949ffb5 => generated 302 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:45.548366 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1aa161-69a6-430e-9dd1-111120e51048 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:45.654090 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:45.662773 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1aa161-69a6-430e-9dd1-111120e51048 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:45.681002 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:45.713351 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf33da-e899-4578-9ec3-616243079f78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:45.714225 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf33da-e899-4578-9ec3-616243079f78 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:45.714504 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf33da-e899-4578-9ec3-616243079f78 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:45.714773 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acf33da-e899-4578-9ec3-616243079f78 None admin] PATH_INFO: `/v3/groups/df89dd397b0e496fad775ebf220818d2` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:45.716067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf33da-e899-4578-9ec3-616243079f78 None admin] RBAC: Authorizing `identity:get_group(group_id=df89dd397b0e496fad775ebf220818d2)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:45.717518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acf33da-e899-4578-9ec3-616243079f78 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:45.719466 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 184/369] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 23 16:53:45 2022] GET /identity/v3/groups/df89dd397b0e496fad775ebf220818d2 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:45.720411 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf33da-e899-4578-9ec3-616243079f78 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:45.740872 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaf28d7-e469-494d-b274-e3aa4e336bde None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:45.777711 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34bef3-36a5-4172-8393-f2fe7ef0c663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:45.779591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34bef3-36a5-4172-8393-f2fe7ef0c663 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:45.779978 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34bef3-36a5-4172-8393-f2fe7ef0c663 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:45.780323 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b34bef3-36a5-4172-8393-f2fe7ef0c663 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:45.849405 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acf33da-e899-4578-9ec3-616243079f78 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:45.888755 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b34bef3-36a5-4172-8393-f2fe7ef0c663 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:45.890244 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b34bef3-36a5-4172-8393-f2fe7ef0c663 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:45.898111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f91e84b-3d80-4fdb-82cf-cf0bc30c2fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:45.899259 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91e84b-3d80-4fdb-82cf-cf0bc30c2fbc None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:45.899735 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91e84b-3d80-4fdb-82cf-cf0bc30c2fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:45.900249 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f91e84b-3d80-4fdb-82cf-cf0bc30c2fbc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:45.914680 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f91e84b-3d80-4fdb-82cf-cf0bc30c2fbc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:45.926217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-1f91e84b-3d80-4fdb-82cf-cf0bc30c2fbc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:53:45.926217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:45.926217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:45.926217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:45.926217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:45.926217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:45.926217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:45.926217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:45.926217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:45.926217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:45.926217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:45.926217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:45.926217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:45.926217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:45.926217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:45.926217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:45.927192 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:45.927192 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:45.927192 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:45.927192 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:45.927192 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:45.927192 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:45.927192 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:45.927192 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:45.927192 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:45.927192 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:45.927192 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:45.927192 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:45.927192 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:45.927192 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:45.927192 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:45.927192 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:45.927971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:45.927971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:45.927971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:45.927971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:45.927971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:45.927971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:45.927971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:45.927971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:45.927971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:45.927971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:45.927971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:53:45.927971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:45.930512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 185/370] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:53:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:45.931760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f91e84b-3d80-4fdb-82cf-cf0bc30c2fbc None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:45.954452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:45.995567 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:46.030818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 186/371] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:53:46.031708 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34bef3-36a5-4172-8393-f2fe7ef0c663 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:46.063938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f91e84b-3d80-4fdb-82cf-cf0bc30c2fbc None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:46.102663 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5f2b7-711e-44d9-a39e-c883258b8985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:46.103692 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5f2b7-711e-44d9-a39e-c883258b8985 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:46.104209 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5f2b7-711e-44d9-a39e-c883258b8985 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:46.105724 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec5f2b7-711e-44d9-a39e-c883258b8985 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:46.106860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5f2b7-711e-44d9-a39e-c883258b8985 None admin] RBAC: Adding query filter params (group.id=df89dd397b0e496fad775ebf220818d2, role.id=72cc26a1a66c4fc183b39fad4cb38431, scope.project.id=4206726f0af54455a4cb03012949ffb5) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:46.107414 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5f2b7-711e-44d9-a39e-c883258b8985 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:46.109099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec5f2b7-711e-44d9-a39e-c883258b8985 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:46.116078 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 186/372] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 23 16:53:45 2022] GET /identity/v3/role_assignments?group.id=df89dd397b0e496fad775ebf220818d2&scope.project.id=4206726f0af54455a4cb03012949ffb5&role.id=72cc26a1a66c4fc183b39fad4cb38431 => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:46.117950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5f2b7-711e-44d9-a39e-c883258b8985 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:46.157894 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b34bef3-36a5-4172-8393-f2fe7ef0c663 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:46.195208 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631b7ae-59d1-4d7d-a44a-fee1dd8b0f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:46.195892 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631b7ae-59d1-4d7d-a44a-fee1dd8b0f70 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:46.196170 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631b7ae-59d1-4d7d-a44a-fee1dd8b0f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:46.196436 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1631b7ae-59d1-4d7d-a44a-fee1dd8b0f70 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:46.197182 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631b7ae-59d1-4d7d-a44a-fee1dd8b0f70 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:46.197484 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631b7ae-59d1-4d7d-a44a-fee1dd8b0f70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:46.198850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1631b7ae-59d1-4d7d-a44a-fee1dd8b0f70 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:46.208841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 187/373] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:46.220083 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631b7ae-59d1-4d7d-a44a-fee1dd8b0f70 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:46.236297 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:46.301728 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec5f2b7-711e-44d9-a39e-c883258b8985 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:46.319833 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:46.358059 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88484ba2-4a61-46a5-84a4-5d18bd3535ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:46.359263 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88484ba2-4a61-46a5-84a4-5d18bd3535ee None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:46.359713 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88484ba2-4a61-46a5-84a4-5d18bd3535ee None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:46.360139 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88484ba2-4a61-46a5-84a4-5d18bd3535ee None admin] PATH_INFO: `/v3/projects/4206726f0af54455a4cb03012949ffb5` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:46.361762 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88484ba2-4a61-46a5-84a4-5d18bd3535ee None admin] RBAC: Authorizing `identity:get_project(project_id=4206726f0af54455a4cb03012949ffb5)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:46.364109 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88484ba2-4a61-46a5-84a4-5d18bd3535ee None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:46.366741 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 187/374] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:46 2022] GET /identity/v3/projects/4206726f0af54455a4cb03012949ffb5 => generated 302 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:46.372904 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88484ba2-4a61-46a5-84a4-5d18bd3535ee None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:46.459669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1631b7ae-59d1-4d7d-a44a-fee1dd8b0f70 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:46.516063 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcefa21-cba8-4097-9b3d-d3ce94c60e46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:46.516712 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcefa21-cba8-4097-9b3d-d3ce94c60e46 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:46.517002 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcefa21-cba8-4097-9b3d-d3ce94c60e46 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:46.517238 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcefa21-cba8-4097-9b3d-d3ce94c60e46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:46.519211 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88484ba2-4a61-46a5-84a4-5d18bd3535ee None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:46.557917 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e5ede-ace2-4726-893d-0346d2d473c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:46.559305 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e5ede-ace2-4726-893d-0346d2d473c4 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:46.559824 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e5ede-ace2-4726-893d-0346d2d473c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:46.560368 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e5ede-ace2-4726-893d-0346d2d473c4 None admin] PATH_INFO: `/v3/groups/df89dd397b0e496fad775ebf220818d2` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:46.561898 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e5ede-ace2-4726-893d-0346d2d473c4 None admin] RBAC: Authorizing `identity:get_group(group_id=df89dd397b0e496fad775ebf220818d2)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:46.563700 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e5ede-ace2-4726-893d-0346d2d473c4 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:46.565996 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 188/375] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 23 16:53:46 2022] GET /identity/v3/groups/df89dd397b0e496fad775ebf220818d2 => generated 226 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:46.572031 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e5ede-ace2-4726-893d-0346d2d473c4 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:46.596285 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcefa21-cba8-4097-9b3d-d3ce94c60e46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:46.597796 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcefa21-cba8-4097-9b3d-d3ce94c60e46 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:46.606359 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:46.634376 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:46.696295 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e5ede-ace2-4726-893d-0346d2d473c4 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:46.700743 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 188/376] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:53:46.709736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcefa21-cba8-4097-9b3d-d3ce94c60e46 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:46.738363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47bdc15-1353-4faf-8212-6001ee87e523 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:46.739882 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bdc15-1353-4faf-8212-6001ee87e523 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:46.740388 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bdc15-1353-4faf-8212-6001ee87e523 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:46.740826 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47bdc15-1353-4faf-8212-6001ee87e523 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:46.741801 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47bdc15-1353-4faf-8212-6001ee87e523 None admin] RBAC: Adding query filter params (group.id=df89dd397b0e496fad775ebf220818d2, role.id=6c30aca17fad4c4fa40b9461c5bda291, scope.project.id=4206726f0af54455a4cb03012949ffb5) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:46.742256 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47bdc15-1353-4faf-8212-6001ee87e523 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:46.743902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47bdc15-1353-4faf-8212-6001ee87e523 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:46.750468 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 189/377] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 23 16:53:46 2022] GET /identity/v3/role_assignments?group.id=df89dd397b0e496fad775ebf220818d2&scope.project.id=4206726f0af54455a4cb03012949ffb5&role.id=6c30aca17fad4c4fa40b9461c5bda291 => generated 260 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:46.880949 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcefa21-cba8-4097-9b3d-d3ce94c60e46 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:46.889998 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:46.919400 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194aba4f-2565-42bf-b8eb-1ae704e1253b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:46.920296 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194aba4f-2565-42bf-b8eb-1ae704e1253b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:46.920556 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194aba4f-2565-42bf-b8eb-1ae704e1253b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:46.920823 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194aba4f-2565-42bf-b8eb-1ae704e1253b None admin] PATH_INFO: `/v3/projects/841c10778a1544ddb64ccc3d96fecf42` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:46.922015 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194aba4f-2565-42bf-b8eb-1ae704e1253b None admin] RBAC: Authorizing `identity:get_project(project_id=841c10778a1544ddb64ccc3d96fecf42)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:46.923502 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194aba4f-2565-42bf-b8eb-1ae704e1253b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:46.925498 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 189/378] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:46 2022] GET /identity/v3/projects/841c10778a1544ddb64ccc3d96fecf42 => generated 344 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:46.931130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47bdc15-1353-4faf-8212-6001ee87e523 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:47.069637 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47bdc15-1353-4faf-8212-6001ee87e523 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:47.079821 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:47.106081 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc437d8c-c947-4c99-803f-f4d15d0a6671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:47.106974 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc437d8c-c947-4c99-803f-f4d15d0a6671 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:47.107258 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc437d8c-c947-4c99-803f-f4d15d0a6671 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:47.107526 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc437d8c-c947-4c99-803f-f4d15d0a6671 None admin] PATH_INFO: `/v3/groups/d79faf80a5004c57b2ced96c4bc51819` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:47.112903 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc437d8c-c947-4c99-803f-f4d15d0a6671 None admin] RBAC: Authorizing `identity:get_group(group_id=d79faf80a5004c57b2ced96c4bc51819)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:47.114399 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc437d8c-c947-4c99-803f-f4d15d0a6671 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:47.116578 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 190/379] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 23 16:53:46 2022] GET /identity/v3/groups/d79faf80a5004c57b2ced96c4bc51819 => generated 229 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:47.122377 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194aba4f-2565-42bf-b8eb-1ae704e1253b None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:47.253767 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194aba4f-2565-42bf-b8eb-1ae704e1253b None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:47.261760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278ea3e-8e18-44b1-b65e-1b3458223020 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:47.262358 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278ea3e-8e18-44b1-b65e-1b3458223020 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:47.262829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f278ea3e-8e18-44b1-b65e-1b3458223020 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:47.265809 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 191/380] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:47.270561 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44ad30-8387-4a58-bb12-e46c69df67df None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:47.271219 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44ad30-8387-4a58-bb12-e46c69df67df None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:47.271722 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e44ad30-8387-4a58-bb12-e46c69df67df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:47.289583 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041bbe0-2f81-451c-aa69-4903c0d1d3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:47.290253 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041bbe0-2f81-451c-aa69-4903c0d1d3cb None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:47.290534 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041bbe0-2f81-451c-aa69-4903c0d1d3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:47.290807 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4041bbe0-2f81-451c-aa69-4903c0d1d3cb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:47.291729 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041bbe0-2f81-451c-aa69-4903c0d1d3cb None admin] RBAC: Adding query filter params (group.id=d79faf80a5004c57b2ced96c4bc51819, role.id=17bbe7cc1c3641509de5e9dca8534102, scope.project.id=841c10778a1544ddb64ccc3d96fecf42) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:47.292071 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041bbe0-2f81-451c-aa69-4903c0d1d3cb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:47.293481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4041bbe0-2f81-451c-aa69-4903c0d1d3cb None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:47.300640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 190/381] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 23 16:53:47 2022] GET /identity/v3/role_assignments?group.id=d79faf80a5004c57b2ced96c4bc51819&scope.project.id=841c10778a1544ddb64ccc3d96fecf42&role.id=17bbe7cc1c3641509de5e9dca8534102 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:47.320314 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-3e44ad30-8387-4a58-bb12-e46c69df67df None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:47.322016 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-3e44ad30-8387-4a58-bb12-e46c69df67df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:47.431867 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 192/382] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 16:53:47.466748 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041bbe0-2f81-451c-aa69-4903c0d1d3cb None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:47.529477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:47.604942 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4041bbe0-2f81-451c-aa69-4903c0d1d3cb None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:47.623103 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922974b-6d9c-4b5f-9824-cf251adc4f75 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:47.623619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922974b-6d9c-4b5f-9824-cf251adc4f75 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:47.623715 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c922974b-6d9c-4b5f-9824-cf251adc4f75 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:47.628608 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 193/383] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:47 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:47.635502 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec37970-edcf-402a-a225-0d2921272f1d None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:47.635794 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec37970-edcf-402a-a225-0d2921272f1d None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:47.636072 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec37970-edcf-402a-a225-0d2921272f1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:47.642867 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49d74a-f8e1-40ac-b150-50e879ecea2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:47.643921 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49d74a-f8e1-40ac-b150-50e879ecea2b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:47.644346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49d74a-f8e1-40ac-b150-50e879ecea2b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:47.644761 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49d74a-f8e1-40ac-b150-50e879ecea2b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:47.668438 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-dec37970-edcf-402a-a225-0d2921272f1d None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:47.669599 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-dec37970-edcf-402a-a225-0d2921272f1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:47.671873 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49d74a-f8e1-40ac-b150-50e879ecea2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:47.679904 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-ef49d74a-f8e1-40ac-b150-50e879ecea2b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:47.679904 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:47.679904 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:47.679904 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:47.679904 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:47.679904 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:47.679904 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:47.679904 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:47.679904 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:47.679904 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:47.679904 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:47.679904 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:47.679904 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:47.679904 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:47.679904 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:47.679904 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:47.680841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:47.680841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:47.680841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:47.680841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:47.680841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:47.680841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:47.680841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:47.680841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:47.680841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:47.680841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:47.680841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:47.680841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:47.680841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:47.680841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:47.680841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:47.680841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:47.681627 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:47.681627 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:47.681627 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:47.681627 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:47.681627 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:47.681627 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:47.681627 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:47.681627 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:47.681627 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:47.681627 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:47.681627 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:47.681627 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:47.681627 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 191/384] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:47.687204 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49d74a-f8e1-40ac-b150-50e879ecea2b None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:47.808842 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 194/385] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 16:53:47.837038 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec37970-edcf-402a-a225-0d2921272f1d None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:47.883426 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:47.897416 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:47.905071 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49d74a-f8e1-40ac-b150-50e879ecea2b None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:47.959110 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec37970-edcf-402a-a225-0d2921272f1d None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:48.001591 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe550cd-0323-484d-87ab-c8cf23b57004 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:48.002900 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe550cd-0323-484d-87ab-c8cf23b57004 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:48.003368 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe550cd-0323-484d-87ab-c8cf23b57004 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:48.003874 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe550cd-0323-484d-87ab-c8cf23b57004 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:48.012172 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffcb09-72a8-4bf2-9492-d21f5e1eec16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:48.012859 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffcb09-72a8-4bf2-9492-d21f5e1eec16 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:48.013147 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffcb09-72a8-4bf2-9492-d21f5e1eec16 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:48.013422 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ffcb09-72a8-4bf2-9492-d21f5e1eec16 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:48.014184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ffcb09-72a8-4bf2-9492-d21f5e1eec16 None admin] RBAC: Adding query filter params (name=member) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:48.014494 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ffcb09-72a8-4bf2-9492-d21f5e1eec16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:48.015908 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ffcb09-72a8-4bf2-9492-d21f5e1eec16 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:48.017573 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe550cd-0323-484d-87ab-c8cf23b57004 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:48.028018 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-1fe550cd-0323-484d-87ab-c8cf23b57004 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:53:48.028018 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:48.028018 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:48.028018 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:48.028018 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:48.028018 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:48.028018 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:48.028018 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:48.028018 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:48.028018 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:48.028018 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:48.028018 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:48.028018 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:48.028018 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:48.028018 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:48.028018 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:48.028948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:48.028948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:48.028948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:48.028948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:48.028948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:48.028948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:48.028948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:48.028948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:48.028948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:48.028948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:48.028948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:48.028948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:48.028948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:48.028948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:48.028948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:48.028948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:48.029734 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:48.029734 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:48.029734 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:48.029734 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:48.029734 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:48.029734 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:48.029734 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:48.029734 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:48.029734 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:48.029734 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:48.029734 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:53:48.029734 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:48.035543 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 195/386] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 23 16:53:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:53:48.039622 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe550cd-0323-484d-87ab-c8cf23b57004 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:48.041385 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 192/387] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:48.047895 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffcb09-72a8-4bf2-9492-d21f5e1eec16 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:48.192805 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ffcb09-72a8-4bf2-9492-d21f5e1eec16 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:48.227032 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d288b-2d0b-4165-be59-8425b804f2d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:48.228001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d288b-2d0b-4165-be59-8425b804f2d2 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:48.228376 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d288b-2d0b-4165-be59-8425b804f2d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:48.228763 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496d288b-2d0b-4165-be59-8425b804f2d2 None admin] PATH_INFO: `/v3/groups/df89dd397b0e496fad775ebf220818d2` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:48.230032 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d288b-2d0b-4165-be59-8425b804f2d2 None admin] RBAC: Authorizing `identity:get_group(group_id=df89dd397b0e496fad775ebf220818d2)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:48.231783 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496d288b-2d0b-4165-be59-8425b804f2d2 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:48.233943 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 193/388] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 23 16:53:48 2022] GET /identity/v3/groups/df89dd397b0e496fad775ebf220818d2 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:48.239130 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d288b-2d0b-4165-be59-8425b804f2d2 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:48.276301 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:48.276488 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:48.324886 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe550cd-0323-484d-87ab-c8cf23b57004 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:48.353120 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496d288b-2d0b-4165-be59-8425b804f2d2 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:48.391778 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b1a5f-6048-4c9e-b8ff-fae1bd7b4ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:48.392703 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b1a5f-6048-4c9e-b8ff-fae1bd7b4ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:48.392983 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b1a5f-6048-4c9e-b8ff-fae1bd7b4ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:48.393253 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17b1a5f-6048-4c9e-b8ff-fae1bd7b4ee4 None admin] PATH_INFO: `/v3/projects/f29295d0fb6141208e3a094c3be29944` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:48.394488 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b1a5f-6048-4c9e-b8ff-fae1bd7b4ee4 None admin] RBAC: Authorizing `identity:get_project(project_id=f29295d0fb6141208e3a094c3be29944)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:48.397350 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17b1a5f-6048-4c9e-b8ff-fae1bd7b4ee4 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:48.399520 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 194/389] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:48 2022] GET /identity/v3/projects/f29295d0fb6141208e3a094c3be29944 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:48.407673 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b1a5f-6048-4c9e-b8ff-fae1bd7b4ee4 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:48.441828 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab11d5-d1f6-4179-863c-b34b93cae103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:48.442779 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab11d5-d1f6-4179-863c-b34b93cae103 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:48.443262 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab11d5-d1f6-4179-863c-b34b93cae103 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:48.443563 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bab11d5-d1f6-4179-863c-b34b93cae103 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:48.444592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab11d5-d1f6-4179-863c-b34b93cae103 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:48.445016 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab11d5-d1f6-4179-863c-b34b93cae103 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:48.447001 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bab11d5-d1f6-4179-863c-b34b93cae103 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:48.466134 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 196/390] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:53:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:48.473101 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab11d5-d1f6-4179-863c-b34b93cae103 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:48.596129 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17b1a5f-6048-4c9e-b8ff-fae1bd7b4ee4 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:48.614272 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bab11d5-d1f6-4179-863c-b34b93cae103 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:48.614986 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:48.646302 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852c71a-0a9c-4b39-b7ce-2cf864351186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:48.647136 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852c71a-0a9c-4b39-b7ce-2cf864351186 None admin] REQUEST_METHOD: `PUT` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:48.647525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852c71a-0a9c-4b39-b7ce-2cf864351186 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:48.647808 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7852c71a-0a9c-4b39-b7ce-2cf864351186 None admin] PATH_INFO: `/v3/projects/f29295d0fb6141208e3a094c3be29944/groups/df89dd397b0e496fad775ebf220818d2/roles/72cc26a1a66c4fc183b39fad4cb38431` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:48.649758 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852c71a-0a9c-4b39-b7ce-2cf864351186 None admin] RBAC: Authorizing `identity:create_grant(project_id=f29295d0fb6141208e3a094c3be29944, group_id=df89dd397b0e496fad775ebf220818d2, role_id=72cc26a1a66c4fc183b39fad4cb38431)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:48.652266 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7852c71a-0a9c-4b39-b7ce-2cf864351186 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:48.656830 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e9086-42f9-44ea-adce-d2c2ff3e45c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:48.657699 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e9086-42f9-44ea-adce-d2c2ff3e45c4 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:48.657977 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e9086-42f9-44ea-adce-d2c2ff3e45c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:48.658238 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e9086-42f9-44ea-adce-d2c2ff3e45c4 None admin] PATH_INFO: `/v3/groups/df89dd397b0e496fad775ebf220818d2` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:48.661225 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e9086-42f9-44ea-adce-d2c2ff3e45c4 None admin] RBAC: Authorizing `identity:get_group(group_id=df89dd397b0e496fad775ebf220818d2)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:48.662991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-272e9086-42f9-44ea-adce-d2c2ff3e45c4 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:48.664818 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 197/391] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 23 16:53:48 2022] GET /identity/v3/groups/df89dd397b0e496fad775ebf220818d2 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:48.671613 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e9086-42f9-44ea-adce-d2c2ff3e45c4 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:48.678064 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 195/392] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 23 16:53:48 2022] PUT /identity/v3/projects/f29295d0fb6141208e3a094c3be29944/groups/df89dd397b0e496fad775ebf220818d2/roles/72cc26a1a66c4fc183b39fad4cb38431 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:53:48.715333 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:48.830660 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e9086-42f9-44ea-adce-d2c2ff3e45c4 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:48.879706 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea55e2e4-c220-4484-90e5-b8628a7bebdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:48.880694 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55e2e4-c220-4484-90e5-b8628a7bebdf None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:48.880979 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55e2e4-c220-4484-90e5-b8628a7bebdf None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:48.881254 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea55e2e4-c220-4484-90e5-b8628a7bebdf None admin] PATH_INFO: `/v3/projects/f29295d0fb6141208e3a094c3be29944` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:48.883511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55e2e4-c220-4484-90e5-b8628a7bebdf None admin] RBAC: Authorizing `identity:get_project(project_id=f29295d0fb6141208e3a094c3be29944)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:48.885291 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea55e2e4-c220-4484-90e5-b8628a7bebdf None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:48.887055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 198/393] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:48 2022] GET /identity/v3/projects/f29295d0fb6141208e3a094c3be29944 => generated 298 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:48.893186 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852c71a-0a9c-4b39-b7ce-2cf864351186 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:49.067633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7852c71a-0a9c-4b39-b7ce-2cf864351186 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:49.121634 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2e5b5-d74b-4065-abbe-7ee4a3f310c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:49.122545 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2e5b5-d74b-4065-abbe-7ee4a3f310c3 None admin] REQUEST_METHOD: `PUT` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:49.122823 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2e5b5-d74b-4065-abbe-7ee4a3f310c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:49.123117 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2e5b5-d74b-4065-abbe-7ee4a3f310c3 None admin] PATH_INFO: `/v3/projects/f29295d0fb6141208e3a094c3be29944/groups/df89dd397b0e496fad775ebf220818d2/roles/6c30aca17fad4c4fa40b9461c5bda291` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:49.125392 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2e5b5-d74b-4065-abbe-7ee4a3f310c3 None admin] RBAC: Authorizing `identity:create_grant(project_id=f29295d0fb6141208e3a094c3be29944, group_id=df89dd397b0e496fad775ebf220818d2, role_id=6c30aca17fad4c4fa40b9461c5bda291)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:49.126846 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2e5b5-d74b-4065-abbe-7ee4a3f310c3 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:49.151182 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 196/394] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 23 16:53:48 2022] PUT /identity/v3/projects/f29295d0fb6141208e3a094c3be29944/groups/df89dd397b0e496fad775ebf220818d2/roles/6c30aca17fad4c4fa40b9461c5bda291 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:53:49.351420 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af747d0-4474-4bad-8d97-41a2b5e97a51 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:49.351824 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af747d0-4474-4bad-8d97-41a2b5e97a51 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:49.351998 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af747d0-4474-4bad-8d97-41a2b5e97a51 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:49.353692 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 199/395] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:49.359632 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879d8d5-7ccc-4cc3-8f15-cb848b651f60 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:49.359949 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879d8d5-7ccc-4cc3-8f15-cb848b651f60 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:49.360196 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a879d8d5-7ccc-4cc3-8f15-cb848b651f60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:49.384313 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-a879d8d5-7ccc-4cc3-8f15-cb848b651f60 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:49.385538 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-a879d8d5-7ccc-4cc3-8f15-cb848b651f60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:49.402624 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:49.478940 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 197/396] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:49.501843 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af747d0-4474-4bad-8d97-41a2b5e97a51 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:49.604775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:49.619070 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af747d0-4474-4bad-8d97-41a2b5e97a51 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:49.641712 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998af79-0cb8-48f8-9533-4ca1256f9495 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:49.642225 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998af79-0cb8-48f8-9533-4ca1256f9495 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:49.642342 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a998af79-0cb8-48f8-9533-4ca1256f9495 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:49.643812 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 198/397] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:53:49.649262 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221b000-130b-4d68-92fe-0c2da3b47738 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:49.649550 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221b000-130b-4d68-92fe-0c2da3b47738 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:49.649830 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4221b000-130b-4d68-92fe-0c2da3b47738 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:49.664835 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c0348-43df-4363-889e-5ee1b3ce904d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:49.666244 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c0348-43df-4363-889e-5ee1b3ce904d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:49.666678 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c0348-43df-4363-889e-5ee1b3ce904d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:49.667111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63c0348-43df-4363-889e-5ee1b3ce904d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:49.670850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-4221b000-130b-4d68-92fe-0c2da3b47738 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:49.670850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-4221b000-130b-4d68-92fe-0c2da3b47738 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:49.692344 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63c0348-43df-4363-889e-5ee1b3ce904d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:49.700753 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-c63c0348-43df-4363-889e-5ee1b3ce904d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:49.700753 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:49.700753 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:49.700753 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:49.700753 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:49.700753 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:49.700753 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:49.700753 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:49.700753 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:49.700753 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:49.700753 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:49.700753 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:49.700753 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:49.700753 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:49.700753 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:49.700753 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:49.701976 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:49.701976 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:49.701976 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:49.701976 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:49.701976 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:49.701976 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:49.701976 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:49.701976 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:49.701976 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:49.701976 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:49.701976 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:49.701976 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:49.701976 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:49.701976 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:49.701976 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:49.701976 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:49.702978 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:49.702978 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:49.702978 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:49.702978 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:49.702978 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:49.702978 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:49.702978 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:49.702978 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:49.702978 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:49.702978 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:49.702978 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:49.702978 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:49.702978 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 200/398] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:49.708960 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c0348-43df-4363-889e-5ee1b3ce904d None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:49.801612 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 199/399] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 16:53:49.863276 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63c0348-43df-4363-889e-5ee1b3ce904d None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:49.869274 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4221b000-130b-4d68-92fe-0c2da3b47738 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:49.905867 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92592468-fb31-47a4-8f7c-32a269122520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:49.906837 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92592468-fb31-47a4-8f7c-32a269122520 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:49.907287 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92592468-fb31-47a4-8f7c-32a269122520 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:49.907693 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92592468-fb31-47a4-8f7c-32a269122520 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:49.908927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92592468-fb31-47a4-8f7c-32a269122520 None admin] RBAC: Adding query filter params (name=member) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:49.909370 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92592468-fb31-47a4-8f7c-32a269122520 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:49.910889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92592468-fb31-47a4-8f7c-32a269122520 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:49.921242 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 201/400] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:49.927886 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92592468-fb31-47a4-8f7c-32a269122520 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:49.945507 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:50.037981 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:50.082584 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92592468-fb31-47a4-8f7c-32a269122520 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:50.105066 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4221b000-130b-4d68-92fe-0c2da3b47738 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:50.142333 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b75a6-31a7-4cd0-b1e5-272119791222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:50.144680 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b75a6-31a7-4cd0-b1e5-272119791222 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:50.147324 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b75a6-31a7-4cd0-b1e5-272119791222 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:50.147796 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3b75a6-31a7-4cd0-b1e5-272119791222 None admin] PATH_INFO: `/v3/groups/df89dd397b0e496fad775ebf220818d2` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:50.151629 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b75a6-31a7-4cd0-b1e5-272119791222 None admin] RBAC: Authorizing `identity:get_group(group_id=df89dd397b0e496fad775ebf220818d2)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:50.153413 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3b75a6-31a7-4cd0-b1e5-272119791222 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:50.155620 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 202/401] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 23 16:53:49 2022] GET /identity/v3/groups/df89dd397b0e496fad775ebf220818d2 => generated 226 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:50.163353 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b75a6-31a7-4cd0-b1e5-272119791222 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:50.167098 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ce2a74-dff1-4958-83ac-c00e42651bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:50.167925 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce2a74-dff1-4958-83ac-c00e42651bfc None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:50.168200 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce2a74-dff1-4958-83ac-c00e42651bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:50.168467 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ce2a74-dff1-4958-83ac-c00e42651bfc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:50.182869 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ce2a74-dff1-4958-83ac-c00e42651bfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:50.191433 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-66ce2a74-dff1-4958-83ac-c00e42651bfc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:53:50.191433 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:50.191433 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:50.191433 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:50.191433 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:50.191433 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:50.191433 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:50.191433 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:50.191433 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:50.191433 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:50.191433 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:50.191433 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:50.191433 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:50.191433 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:50.191433 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:50.191433 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:50.192669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:50.192669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:50.192669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:50.192669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:50.192669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:50.192669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:50.192669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:50.192669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:50.192669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:50.192669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:50.192669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:50.192669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:50.192669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:50.192669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:50.192669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:50.192669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:50.193657 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:50.193657 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:50.193657 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:50.193657 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:50.193657 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:50.193657 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:50.193657 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:50.193657 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:50.193657 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:50.193657 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:50.193657 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:53:50.193657 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:50.196136 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 200/402] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 23 16:53:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:53:50.197522 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d02127-ebb8-4185-b6ae-6dcaaa984536 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:50.197956 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d02127-ebb8-4185-b6ae-6dcaaa984536 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:50.198391 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d02127-ebb8-4185-b6ae-6dcaaa984536 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:50.204439 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 201/403] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:50.206307 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d02127-ebb8-4185-b6ae-6dcaaa984536 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:50.276592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:50.307750 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3b75a6-31a7-4cd0-b1e5-272119791222 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:50.312545 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d02127-ebb8-4185-b6ae-6dcaaa984536 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:50.356141 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c9dc4-3b1c-4c76-8a74-4255a0dd55c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:50.356930 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c9dc4-3b1c-4c76-8a74-4255a0dd55c6 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:50.357324 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c9dc4-3b1c-4c76-8a74-4255a0dd55c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:50.357681 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c27c9dc4-3b1c-4c76-8a74-4255a0dd55c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:50.358588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c9dc4-3b1c-4c76-8a74-4255a0dd55c6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:50.359060 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c9dc4-3b1c-4c76-8a74-4255a0dd55c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:50.360626 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c27c9dc4-3b1c-4c76-8a74-4255a0dd55c6 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:50.361772 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7493f4df-05c5-40ee-81f9-e0ec8fe9f385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:50.363053 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493f4df-05c5-40ee-81f9-e0ec8fe9f385 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:50.363829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493f4df-05c5-40ee-81f9-e0ec8fe9f385 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:50.364310 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7493f4df-05c5-40ee-81f9-e0ec8fe9f385 None admin] PATH_INFO: `/v3/projects/4206726f0af54455a4cb03012949ffb5` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:50.365845 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7493f4df-05c5-40ee-81f9-e0ec8fe9f385 None admin] RBAC: Authorizing `identity:get_project(project_id=4206726f0af54455a4cb03012949ffb5)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:50.367254 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:50.368021 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7493f4df-05c5-40ee-81f9-e0ec8fe9f385 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:50.370225 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 203/404] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:50 2022] GET /identity/v3/projects/4206726f0af54455a4cb03012949ffb5 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:50.372596 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe9c30-d0d2-4c0b-a8bd-c58880d7c612 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:50.372887 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe9c30-d0d2-4c0b-a8bd-c58880d7c612 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:50.373164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffe9c30-d0d2-4c0b-a8bd-c58880d7c612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:50.375175 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 202/405] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:53:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:50.382212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c9dc4-3b1c-4c76-8a74-4255a0dd55c6 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:50.397999 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-dffe9c30-d0d2-4c0b-a8bd-c58880d7c612 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:50.399314 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-dffe9c30-d0d2-4c0b-a8bd-c58880d7c612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:50.478858 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 204/406] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:50.481932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe9c30-d0d2-4c0b-a8bd-c58880d7c612 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:50.503305 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c27c9dc4-3b1c-4c76-8a74-4255a0dd55c6 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:50.543466 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c32322-d709-46fd-b2ca-963474637c21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:50.544525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c32322-d709-46fd-b2ca-963474637c21 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:50.544919 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c32322-d709-46fd-b2ca-963474637c21 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:50.545370 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c32322-d709-46fd-b2ca-963474637c21 None admin] PATH_INFO: `/v3/groups/df89dd397b0e496fad775ebf220818d2` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:50.546681 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c32322-d709-46fd-b2ca-963474637c21 None admin] RBAC: Authorizing `identity:get_group(group_id=df89dd397b0e496fad775ebf220818d2)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:50.548334 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c32322-d709-46fd-b2ca-963474637c21 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:50.550576 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 203/407] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 23 16:53:50 2022] GET /identity/v3/groups/df89dd397b0e496fad775ebf220818d2 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:50.551550 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c32322-d709-46fd-b2ca-963474637c21 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:50.594725 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:50.627719 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffe9c30-d0d2-4c0b-a8bd-c58880d7c612 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:50.629916 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:50.666982 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3829156-ed22-4626-9082-48adb08db810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:50.667792 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3829156-ed22-4626-9082-48adb08db810 None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:50.668066 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3829156-ed22-4626-9082-48adb08db810 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:50.668335 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3829156-ed22-4626-9082-48adb08db810 None admin] PATH_INFO: `/v3/projects/4206726f0af54455a4cb03012949ffb5/groups/df89dd397b0e496fad775ebf220818d2/roles/72cc26a1a66c4fc183b39fad4cb38431` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:50.669378 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c32322-d709-46fd-b2ca-963474637c21 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:50.670498 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3829156-ed22-4626-9082-48adb08db810 None admin] RBAC: Authorizing `identity:create_grant(project_id=4206726f0af54455a4cb03012949ffb5, group_id=df89dd397b0e496fad775ebf220818d2, role_id=72cc26a1a66c4fc183b39fad4cb38431)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:50.672271 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3829156-ed22-4626-9082-48adb08db810 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:50.687242 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 205/408] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 23 16:53:50 2022] PUT /identity/v3/projects/4206726f0af54455a4cb03012949ffb5/groups/df89dd397b0e496fad775ebf220818d2/roles/72cc26a1a66c4fc183b39fad4cb38431 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:53:50.688408 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3829156-ed22-4626-9082-48adb08db810 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:50.709933 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452e569-52a9-47fe-9317-524126b93423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:50.710887 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452e569-52a9-47fe-9317-524126b93423 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:50.711208 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452e569-52a9-47fe-9317-524126b93423 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:50.711482 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c452e569-52a9-47fe-9317-524126b93423 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:50.722949 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c452e569-52a9-47fe-9317-524126b93423 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:50.731402 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-c452e569-52a9-47fe-9317-524126b93423 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:53:50.731402 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:50.731402 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:50.731402 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:50.731402 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:50.731402 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:50.731402 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:50.731402 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:50.731402 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:50.731402 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:50.731402 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:50.731402 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:50.731402 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:50.731402 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:50.731402 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:50.731402 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:50.733247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:50.733247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:50.733247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:50.733247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:50.733247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:50.733247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:50.733247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:50.733247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:50.733247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:50.733247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:50.733247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:50.733247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:50.733247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:50.733247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:50.733247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:50.733247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:50.734857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:50.734857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:50.734857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:50.734857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:50.734857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:50.734857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:50.734857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:50.734857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:50.734857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:50.734857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:50.734857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:53:50.734857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:50.739549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 204/409] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:53:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:50.739549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452e569-52a9-47fe-9317-524126b93423 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:50.807541 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3829156-ed22-4626-9082-48adb08db810 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:50.845701 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d176594-4f76-430b-824b-a9f697b6d3b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:50.846829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d176594-4f76-430b-824b-a9f697b6d3b9 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:50.847200 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d176594-4f76-430b-824b-a9f697b6d3b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:50.847533 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d176594-4f76-430b-824b-a9f697b6d3b9 None admin] PATH_INFO: `/v3/projects/4206726f0af54455a4cb03012949ffb5` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:50.849039 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d176594-4f76-430b-824b-a9f697b6d3b9 None admin] RBAC: Authorizing `identity:get_project(project_id=4206726f0af54455a4cb03012949ffb5)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:50.850662 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d176594-4f76-430b-824b-a9f697b6d3b9 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:50.852623 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 206/410] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:50 2022] GET /identity/v3/projects/4206726f0af54455a4cb03012949ffb5 => generated 302 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:50.853133 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c452e569-52a9-47fe-9317-524126b93423 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:50.858740 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d176594-4f76-430b-824b-a9f697b6d3b9 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:50.868663 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:50.888049 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:50.900001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91817d93-15e1-4a61-b930-32f12752c70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:50.901104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91817d93-15e1-4a61-b930-32f12752c70c None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:50.901569 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91817d93-15e1-4a61-b930-32f12752c70c None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:50.902149 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91817d93-15e1-4a61-b930-32f12752c70c None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:50.903144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91817d93-15e1-4a61-b930-32f12752c70c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:50.903661 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91817d93-15e1-4a61-b930-32f12752c70c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:50.905397 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91817d93-15e1-4a61-b930-32f12752c70c None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:50.920362 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 205/411] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:50.922040 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91817d93-15e1-4a61-b930-32f12752c70c None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:50.982234 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d176594-4f76-430b-824b-a9f697b6d3b9 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:51.021169 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acee38b-dee0-4d77-8b43-eb75b6a56543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:51.021793 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acee38b-dee0-4d77-8b43-eb75b6a56543 None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:51.022140 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acee38b-dee0-4d77-8b43-eb75b6a56543 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:51.022472 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acee38b-dee0-4d77-8b43-eb75b6a56543 None admin] PATH_INFO: `/v3/projects/4206726f0af54455a4cb03012949ffb5/groups/df89dd397b0e496fad775ebf220818d2/roles/6c30aca17fad4c4fa40b9461c5bda291` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:51.025092 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acee38b-dee0-4d77-8b43-eb75b6a56543 None admin] RBAC: Authorizing `identity:create_grant(project_id=4206726f0af54455a4cb03012949ffb5, group_id=df89dd397b0e496fad775ebf220818d2, role_id=6c30aca17fad4c4fa40b9461c5bda291)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:51.026787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acee38b-dee0-4d77-8b43-eb75b6a56543 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:51.036472 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91817d93-15e1-4a61-b930-32f12752c70c None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:51.043886 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 207/412] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 23 16:53:50 2022] PUT /identity/v3/projects/4206726f0af54455a4cb03012949ffb5/groups/df89dd397b0e496fad775ebf220818d2/roles/6c30aca17fad4c4fa40b9461c5bda291 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 16:53:51.098249 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f8944-fd34-4153-9659-f973dc520d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:51.099817 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f8944-fd34-4153-9659-f973dc520d2d None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:51.100306 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f8944-fd34-4153-9659-f973dc520d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:51.100777 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240f8944-fd34-4153-9659-f973dc520d2d None admin] PATH_INFO: `/v3/groups/d79faf80a5004c57b2ced96c4bc51819` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:51.102975 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240f8944-fd34-4153-9659-f973dc520d2d None admin] RBAC: Authorizing `identity:get_group(group_id=d79faf80a5004c57b2ced96c4bc51819)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:51.104823 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240f8944-fd34-4153-9659-f973dc520d2d None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:51.111121 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 206/413] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 23 16:53:50 2022] GET /identity/v3/groups/d79faf80a5004c57b2ced96c4bc51819 => generated 229 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:51.113422 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acee38b-dee0-4d77-8b43-eb75b6a56543 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:51.198201 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:51.248860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acee38b-dee0-4d77-8b43-eb75b6a56543 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:51.298815 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3d4463-97ce-43f1-9e3a-e29d4417c8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:51.299730 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d4463-97ce-43f1-9e3a-e29d4417c8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:51.300008 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d4463-97ce-43f1-9e3a-e29d4417c8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:51.300277 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3d4463-97ce-43f1-9e3a-e29d4417c8c6 None admin] PATH_INFO: `/v3/projects/841c10778a1544ddb64ccc3d96fecf42` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:51.301463 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3d4463-97ce-43f1-9e3a-e29d4417c8c6 None admin] RBAC: Authorizing `identity:get_project(project_id=841c10778a1544ddb64ccc3d96fecf42)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:51.302997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3d4463-97ce-43f1-9e3a-e29d4417c8c6 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:51.305267 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 208/414] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:51 2022] GET /identity/v3/projects/841c10778a1544ddb64ccc3d96fecf42 => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:51.311187 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f8944-fd34-4153-9659-f973dc520d2d None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:51.377170 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:51.424308 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240f8944-fd34-4153-9659-f973dc520d2d None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:51.460538 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d286e4f3-8590-44ba-ac3c-09c6d9c469d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:51.461731 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286e4f3-8590-44ba-ac3c-09c6d9c469d4 None admin] REQUEST_METHOD: `PUT` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:51.462178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286e4f3-8590-44ba-ac3c-09c6d9c469d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:51.462611 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d286e4f3-8590-44ba-ac3c-09c6d9c469d4 None admin] PATH_INFO: `/v3/projects/841c10778a1544ddb64ccc3d96fecf42/groups/d79faf80a5004c57b2ced96c4bc51819/roles/17bbe7cc1c3641509de5e9dca8534102` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:51.464805 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d286e4f3-8590-44ba-ac3c-09c6d9c469d4 None admin] RBAC: Authorizing `identity:create_grant(project_id=841c10778a1544ddb64ccc3d96fecf42, group_id=d79faf80a5004c57b2ced96c4bc51819, role_id=17bbe7cc1c3641509de5e9dca8534102)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:51.466504 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d286e4f3-8590-44ba-ac3c-09c6d9c469d4 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:51.482732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 207/415] 10.222.0.13 () {58 vars in 1442 bytes} [Mon May 23 16:53:51 2022] PUT /identity/v3/projects/841c10778a1544ddb64ccc3d96fecf42/groups/d79faf80a5004c57b2ced96c4bc51819/roles/17bbe7cc1c3641509de5e9dca8534102 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 16:53:51.791378 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f497a2-bb9b-4995-a3c9-ba538adddb66 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:51.791868 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f497a2-bb9b-4995-a3c9-ba538adddb66 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:51.791868 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f497a2-bb9b-4995-a3c9-ba538adddb66 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:51.793236 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 209/416] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:51.799209 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418875ed-c014-4050-bd51-c7ba7b34d3dd None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:51.799710 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418875ed-c014-4050-bd51-c7ba7b34d3dd None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:51.799781 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418875ed-c014-4050-bd51-c7ba7b34d3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:51.818128 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-418875ed-c014-4050-bd51-c7ba7b34d3dd None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:51.819335 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-418875ed-c014-4050-bd51-c7ba7b34d3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:51.894889 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 208/417] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:51.920063 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f497a2-bb9b-4995-a3c9-ba538adddb66 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:51.983944 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe48bd-8a2c-4403-8a63-cf66bff85ea0 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:51.984264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe48bd-8a2c-4403-8a63-cf66bff85ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:51.984499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe48bd-8a2c-4403-8a63-cf66bff85ea0 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:51.986294 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 209/418] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:53:51.992782 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c6b54-095e-47d6-b3a2-7973cdf9acc4 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:51.993070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c6b54-095e-47d6-b3a2-7973cdf9acc4 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:51.993439 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-322c6b54-095e-47d6-b3a2-7973cdf9acc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:52.019461 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-322c6b54-095e-47d6-b3a2-7973cdf9acc4 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:52.021028 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-322c6b54-095e-47d6-b3a2-7973cdf9acc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:52.047770 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f497a2-bb9b-4995-a3c9-ba538adddb66 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:52.113818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:52.117762 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f21f4-4524-4f56-bdd7-361d0f09b25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:52.118786 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f21f4-4524-4f56-bdd7-361d0f09b25c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:52.119085 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f21f4-4524-4f56-bdd7-361d0f09b25c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:52.119350 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6f21f4-4524-4f56-bdd7-361d0f09b25c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:52.123828 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:52.146238 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 210/419] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 16:53:52.151968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6f21f4-4524-4f56-bdd7-361d0f09b25c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:52.167752 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-fd6f21f4-4524-4f56-bdd7-361d0f09b25c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:52.167752 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:52.167752 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:52.167752 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:52.167752 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:52.167752 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:52.167752 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:52.167752 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:52.167752 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:52.167752 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:52.167752 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:52.167752 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:52.167752 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:52.167752 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:52.167752 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:52.167752 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:52.169872 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:52.169872 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:52.169872 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:52.169872 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:52.169872 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:52.169872 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:52.169872 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:52.169872 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:52.169872 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:52.169872 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:52.169872 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:52.169872 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:52.169872 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:52.169872 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:52.169872 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:52.169872 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:52.171800 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:52.171800 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:52.171800 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:52.171800 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:52.171800 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:52.171800 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:52.171800 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:52.171800 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:52.171800 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:52.171800 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:52.171800 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:52.171800 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:52.173207 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c6b54-095e-47d6-b3a2-7973cdf9acc4 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:52.173355 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 210/420] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:53:52.174997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f21f4-4524-4f56-bdd7-361d0f09b25c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:52.339662 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-322c6b54-095e-47d6-b3a2-7973cdf9acc4 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:52.370237 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6f21f4-4524-4f56-bdd7-361d0f09b25c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:52.388772 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a57c65-8653-4ad0-8a28-2a46185114a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:52.389612 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a57c65-8653-4ad0-8a28-2a46185114a3 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:52.389895 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a57c65-8653-4ad0-8a28-2a46185114a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:52.390165 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a57c65-8653-4ad0-8a28-2a46185114a3 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:52.402633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a57c65-8653-4ad0-8a28-2a46185114a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:52.412527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-13a57c65-8653-4ad0-8a28-2a46185114a3 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:53:52.412527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:52.412527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:52.412527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:52.412527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:52.412527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:52.412527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:52.412527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:52.412527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:52.412527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:52.412527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:52.412527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:52.412527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:52.412527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:52.412527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:52.412527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:52.414555 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:52.414555 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:52.414555 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:52.414555 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:52.414555 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:52.414555 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:52.414555 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:52.414555 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:52.414555 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:52.414555 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:52.414555 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:52.414555 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:52.414555 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:52.414555 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:52.414555 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:52.414555 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:52.416515 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:52.416515 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:52.416515 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:52.416515 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:52.416515 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:52.416515 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:52.416515 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:52.416515 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:52.416515 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:52.416515 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:52.416515 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:53:52.416515 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:52.416515 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 211/421] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 23 16:53:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:53:52.422950 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a57c65-8653-4ad0-8a28-2a46185114a3 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:52.436125 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e108735-360a-49ff-8564-2f7792ef1070 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:52.436806 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e108735-360a-49ff-8564-2f7792ef1070 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:52.437080 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e108735-360a-49ff-8564-2f7792ef1070 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:52.437345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e108735-360a-49ff-8564-2f7792ef1070 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:52.438099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e108735-360a-49ff-8564-2f7792ef1070 None admin] RBAC: Adding query filter params (name=member) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:52.438401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e108735-360a-49ff-8564-2f7792ef1070 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:52.439839 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e108735-360a-49ff-8564-2f7792ef1070 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:52.468002 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:52.472760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 211/422] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:52.476974 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e108735-360a-49ff-8564-2f7792ef1070 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:52.588044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:52.589783 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a57c65-8653-4ad0-8a28-2a46185114a3 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:52.665564 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c18872-6a3a-4b66-a509-32c39c0dd691 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:52.666250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c18872-6a3a-4b66-a509-32c39c0dd691 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:52.666532 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c18872-6a3a-4b66-a509-32c39c0dd691 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:52.666807 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c18872-6a3a-4b66-a509-32c39c0dd691 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:52.667587 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c18872-6a3a-4b66-a509-32c39c0dd691 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:52.667895 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c18872-6a3a-4b66-a509-32c39c0dd691 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:52.669278 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c18872-6a3a-4b66-a509-32c39c0dd691 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:52.674499 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e108735-360a-49ff-8564-2f7792ef1070 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:52.691443 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 212/423] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:53:52 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:52.699550 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c18872-6a3a-4b66-a509-32c39c0dd691 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:52.770298 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955f751-d348-49df-88a4-e9582b583f4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:52.771668 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955f751-d348-49df-88a4-e9582b583f4a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:52.771956 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955f751-d348-49df-88a4-e9582b583f4a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:52.772231 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955f751-d348-49df-88a4-e9582b583f4a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:52.867359 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:52.872855 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955f751-d348-49df-88a4-e9582b583f4a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:52.874460 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955f751-d348-49df-88a4-e9582b583f4a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:52.876348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c18872-6a3a-4b66-a509-32c39c0dd691 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:52.934128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:52.968593 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab320d8-d91a-434e-8fba-22aa2726f4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:52.969257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab320d8-d91a-434e-8fba-22aa2726f4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:52.969537 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab320d8-d91a-434e-8fba-22aa2726f4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:52.969807 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab320d8-d91a-434e-8fba-22aa2726f4e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:52.983631 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 212/424] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 16:53:52.989750 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955f751-d348-49df-88a4-e9582b583f4a None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:53.100581 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955f751-d348-49df-88a4-e9582b583f4a None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:53.136210 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab320d8-d91a-434e-8fba-22aa2726f4e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:53.137629 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab320d8-d91a-434e-8fba-22aa2726f4e2 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:53.148252 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa6a86-18b0-46da-b133-ec2d1a6295eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:53.149469 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa6a86-18b0-46da-b133-ec2d1a6295eb None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:53.149944 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa6a86-18b0-46da-b133-ec2d1a6295eb None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:53.150436 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfa6a86-18b0-46da-b133-ec2d1a6295eb None admin] PATH_INFO: `/v3/projects/f29295d0fb6141208e3a094c3be29944` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:53.155673 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa6a86-18b0-46da-b133-ec2d1a6295eb None admin] RBAC: Authorizing `identity:get_project(project_id=f29295d0fb6141208e3a094c3be29944)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:53.157374 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfa6a86-18b0-46da-b133-ec2d1a6295eb None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:53.159683 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 213/425] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:52 2022] GET /identity/v3/projects/f29295d0fb6141208e3a094c3be29944 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:53.165998 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa6a86-18b0-46da-b133-ec2d1a6295eb None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:53.193916 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:53.277706 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfa6a86-18b0-46da-b133-ec2d1a6295eb None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:53.318376 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:53.320546 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbadb5d-80f3-4e2b-b5dc-0add2a9a1f4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:53.321745 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbadb5d-80f3-4e2b-b5dc-0add2a9a1f4e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:53.322257 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbadb5d-80f3-4e2b-b5dc-0add2a9a1f4e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:53.322731 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbadb5d-80f3-4e2b-b5dc-0add2a9a1f4e None admin] PATH_INFO: `/v3/groups/df89dd397b0e496fad775ebf220818d2` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:53.324293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbadb5d-80f3-4e2b-b5dc-0add2a9a1f4e None admin] RBAC: Authorizing `identity:get_group(group_id=df89dd397b0e496fad775ebf220818d2)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:53.326113 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cbadb5d-80f3-4e2b-b5dc-0add2a9a1f4e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:53.326842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 213/426] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:53:53.334104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab320d8-d91a-434e-8fba-22aa2726f4e2 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:53.335317 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 214/427] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 23 16:53:53 2022] GET /identity/v3/groups/df89dd397b0e496fad775ebf220818d2 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:53.339718 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbadb5d-80f3-4e2b-b5dc-0add2a9a1f4e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:53.438827 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:53.450102 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbadb5d-80f3-4e2b-b5dc-0add2a9a1f4e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:53.488336 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f87b589f-f9f6-4a15-b139-473a7c7a4a9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:53.489311 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b589f-f9f6-4a15-b139-473a7c7a4a9e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:53.490860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b589f-f9f6-4a15-b139-473a7c7a4a9e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:53.491327 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f87b589f-f9f6-4a15-b139-473a7c7a4a9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:53.492302 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b589f-f9f6-4a15-b139-473a7c7a4a9e None admin] RBAC: Adding query filter params (group.id=df89dd397b0e496fad775ebf220818d2, role.id=72cc26a1a66c4fc183b39fad4cb38431, scope.project.id=f29295d0fb6141208e3a094c3be29944) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:53.492758 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b589f-f9f6-4a15-b139-473a7c7a4a9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:53.494350 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f87b589f-f9f6-4a15-b139-473a7c7a4a9e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:53.501084 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 215/428] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 23 16:53:53 2022] GET /identity/v3/role_assignments?group.id=df89dd397b0e496fad775ebf220818d2&scope.project.id=f29295d0fb6141208e3a094c3be29944&role.id=72cc26a1a66c4fc183b39fad4cb38431 => generated 611 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:53.530040 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab320d8-d91a-434e-8fba-22aa2726f4e2 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:53.533818 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c7abd-7002-403c-924d-4afaab7108b2 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:53.534300 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c7abd-7002-403c-924d-4afaab7108b2 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:53.534720 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4c7abd-7002-403c-924d-4afaab7108b2 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:53.536661 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 216/429] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:53:53.543821 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbf431-36a2-4ee8-94a9-bcdab0ebaefc None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:53.544299 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbf431-36a2-4ee8-94a9-bcdab0ebaefc None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:53.544719 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dbf431-36a2-4ee8-94a9-bcdab0ebaefc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:53.574787 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8f490-92cf-456e-a4c2-dcb38f2873b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:53.576189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8f490-92cf-456e-a4c2-dcb38f2873b6 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:53.576645 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8f490-92cf-456e-a4c2-dcb38f2873b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:53.577084 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e8f490-92cf-456e-a4c2-dcb38f2873b6 None admin] PATH_INFO: `/v3/projects/f29295d0fb6141208e3a094c3be29944` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:53.578647 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8f490-92cf-456e-a4c2-dcb38f2873b6 None admin] RBAC: Authorizing `identity:get_project(project_id=f29295d0fb6141208e3a094c3be29944)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:53.580382 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e8f490-92cf-456e-a4c2-dcb38f2873b6 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:53.581936 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-70dbf431-36a2-4ee8-94a9-bcdab0ebaefc None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:53.583097 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 214/430] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:53 2022] GET /identity/v3/projects/f29295d0fb6141208e3a094c3be29944 => generated 298 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:53.583249 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-70dbf431-36a2-4ee8-94a9-bcdab0ebaefc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:53.589681 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8f490-92cf-456e-a4c2-dcb38f2873b6 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:53.679729 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:53.720524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e8f490-92cf-456e-a4c2-dcb38f2873b6 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:53.764852 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1afb49-93b1-49b1-8d70-4d2dbd816ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:53.765723 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1afb49-93b1-49b1-8d70-4d2dbd816ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:53.766007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1afb49-93b1-49b1-8d70-4d2dbd816ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:53.766285 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1afb49-93b1-49b1-8d70-4d2dbd816ee9 None admin] PATH_INFO: `/v3/groups/df89dd397b0e496fad775ebf220818d2` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:53.767519 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1afb49-93b1-49b1-8d70-4d2dbd816ee9 None admin] RBAC: Authorizing `identity:get_group(group_id=df89dd397b0e496fad775ebf220818d2)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:53.768968 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1afb49-93b1-49b1-8d70-4d2dbd816ee9 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:53.772075 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 215/432] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 23 16:53:53 2022] GET /identity/v3/groups/df89dd397b0e496fad775ebf220818d2 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:53.772207 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 217/432] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:53.779916 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1afb49-93b1-49b1-8d70-4d2dbd816ee9 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:53.797612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dbf431-36a2-4ee8-94a9-bcdab0ebaefc None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:53.895114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1afb49-93b1-49b1-8d70-4d2dbd816ee9 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:53.925446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:53.940758 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1a9623-fb1f-42fb-9267-e4cfe338743b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:53.941431 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a9623-fb1f-42fb-9267-e4cfe338743b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:53.941714 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a9623-fb1f-42fb-9267-e4cfe338743b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:53.941988 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1a9623-fb1f-42fb-9267-e4cfe338743b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:53.944257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a9623-fb1f-42fb-9267-e4cfe338743b None admin] RBAC: Adding query filter params (group.id=df89dd397b0e496fad775ebf220818d2, role.id=6c30aca17fad4c4fa40b9461c5bda291, scope.project.id=f29295d0fb6141208e3a094c3be29944) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:53.944257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a9623-fb1f-42fb-9267-e4cfe338743b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:53.946547 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1a9623-fb1f-42fb-9267-e4cfe338743b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:53.947952 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:53.949412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dbf431-36a2-4ee8-94a9-bcdab0ebaefc None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:53.953389 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 216/433] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 23 16:53:53 2022] GET /identity/v3/role_assignments?group.id=df89dd397b0e496fad775ebf220818d2&scope.project.id=f29295d0fb6141208e3a094c3be29944&role.id=6c30aca17fad4c4fa40b9461c5bda291 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:53.958459 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f6f77-1f18-483a-95df-1f187733fae1 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:53.958459 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f6f77-1f18-483a-95df-1f187733fae1 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:53.958459 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f6f77-1f18-483a-95df-1f187733fae1 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:53.958459 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 217/434] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:53.964701 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdcd65-09cb-46f0-a228-f337bff6c57d None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:53.965002 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdcd65-09cb-46f0-a228-f337bff6c57d None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:53.965256 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cdcd65-09cb-46f0-a228-f337bff6c57d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:53.987380 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-c2cdcd65-09cb-46f0-a228-f337bff6c57d None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:53.988499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-c2cdcd65-09cb-46f0-a228-f337bff6c57d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:54.000725 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0e4f6-0342-499a-bdc0-bdc3e11ce195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:54.002022 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0e4f6-0342-499a-bdc0-bdc3e11ce195 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:54.002530 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0e4f6-0342-499a-bdc0-bdc3e11ce195 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:54.003005 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b0e4f6-0342-499a-bdc0-bdc3e11ce195 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:54.017087 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b0e4f6-0342-499a-bdc0-bdc3e11ce195 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:54.026222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-08b0e4f6-0342-499a-bdc0-bdc3e11ce195 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:54.026222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:54.026222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:54.026222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:54.026222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:54.026222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:54.026222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:54.026222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:54.026222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:54.026222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:54.026222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:54.026222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:54.026222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:54.026222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:54.026222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:54.026222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:54.027372 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:54.027372 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:54.027372 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:54.027372 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:54.027372 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:54.027372 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:54.027372 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:54.027372 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:54.027372 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:54.027372 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:54.027372 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:54.027372 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:54.027372 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:54.027372 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:54.027372 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:54.027372 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:54.028365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:54.028365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:54.028365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:54.028365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:54.028365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:54.028365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:54.028365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:54.028365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:54.028365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:54.028365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:54.028365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 16:53:54.028365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:54.030847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 218/435] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:53:54.039998 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0e4f6-0342-499a-bdc0-bdc3e11ce195 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:54.070486 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 218/436] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 16:53:54.103515 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cdcd65-09cb-46f0-a228-f337bff6c57d None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:54.177122 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b0e4f6-0342-499a-bdc0-bdc3e11ce195 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:54.217407 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b3f74-a643-41b9-b3d3-46f917111b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:54.218435 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b3f74-a643-41b9-b3d3-46f917111b67 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:54.218952 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b3f74-a643-41b9-b3d3-46f917111b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:54.219414 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5b3f74-a643-41b9-b3d3-46f917111b67 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:54.220359 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b3f74-a643-41b9-b3d3-46f917111b67 None admin] RBAC: Adding query filter params (name=member) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:54.220736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b3f74-a643-41b9-b3d3-46f917111b67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:54.222189 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5b3f74-a643-41b9-b3d3-46f917111b67 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:54.233465 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 219/437] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:53:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:54.241780 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b3f74-a643-41b9-b3d3-46f917111b67 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:54.251849 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cdcd65-09cb-46f0-a228-f337bff6c57d None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:54.255688 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:54.289630 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:54.298965 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5ef50-5917-4f7a-8adb-e8382f9dbb68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:54.299803 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5ef50-5917-4f7a-8adb-e8382f9dbb68 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:54.300081 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5ef50-5917-4f7a-8adb-e8382f9dbb68 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:54.300353 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d5ef50-5917-4f7a-8adb-e8382f9dbb68 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:54.313340 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d5ef50-5917-4f7a-8adb-e8382f9dbb68 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:54.323328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-14d5ef50-5917-4f7a-8adb-e8382f9dbb68 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:53:54.323328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:54.323328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:54.323328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:54.323328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:54.323328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:54.323328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:54.323328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:54.323328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:54.323328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:54.323328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:54.323328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:54.323328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:54.323328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:54.323328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:54.323328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:54.324310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:54.324310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:54.324310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:54.324310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:54.324310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:54.324310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:54.324310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:54.324310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:54.324310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:54.324310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:54.324310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:54.324310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:54.324310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:54.324310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:54.324310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:54.324310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:54.325469 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:54.325469 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:54.325469 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:54.325469 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:54.325469 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:54.325469 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:54.325469 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:54.325469 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:54.325469 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:54.325469 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:54.325469 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:53:54.325469 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:54.325469 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 219/438] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 23 16:53:54 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:54.330574 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5ef50-5917-4f7a-8adb-e8382f9dbb68 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:54.367451 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5b3f74-a643-41b9-b3d3-46f917111b67 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:54.411113 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b906f5-57e4-42b6-aa01-e0d1ddd4d885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:54.411860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b906f5-57e4-42b6-aa01-e0d1ddd4d885 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:54.412143 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b906f5-57e4-42b6-aa01-e0d1ddd4d885 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:54.412472 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b906f5-57e4-42b6-aa01-e0d1ddd4d885 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:54.464098 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d5ef50-5917-4f7a-8adb-e8382f9dbb68 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:54.496235 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b906f5-57e4-42b6-aa01-e0d1ddd4d885 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:54.500253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b906f5-57e4-42b6-aa01-e0d1ddd4d885 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:54.508093 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2804c865-dab4-401b-9a25-61f5a396aa60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:54.509099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804c865-dab4-401b-9a25-61f5a396aa60 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:54.509384 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804c865-dab4-401b-9a25-61f5a396aa60 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:54.509655 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2804c865-dab4-401b-9a25-61f5a396aa60 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:54.510408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804c865-dab4-401b-9a25-61f5a396aa60 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:54.510805 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804c865-dab4-401b-9a25-61f5a396aa60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:54.510905 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:54.512157 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2804c865-dab4-401b-9a25-61f5a396aa60 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:54.522315 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 220/439] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:53:54 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:54.523600 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516571d9-5ad5-4880-8c8b-70816134392e None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:54.523884 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516571d9-5ad5-4880-8c8b-70816134392e None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:54.524160 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516571d9-5ad5-4880-8c8b-70816134392e None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:54.525932 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 221/440] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:54.529221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516571d9-5ad5-4880-8c8b-70816134392e None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:54.586104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:54.607736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 220/441] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:53:54.609040 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a41e6-1c02-44f3-80f9-db95201b4e56 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:54.609327 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a41e6-1c02-44f3-80f9-db95201b4e56 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:54.609599 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482a41e6-1c02-44f3-80f9-db95201b4e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:54.628110 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-482a41e6-1c02-44f3-80f9-db95201b4e56 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:54.629294 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-482a41e6-1c02-44f3-80f9-db95201b4e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:54.648781 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516571d9-5ad5-4880-8c8b-70816134392e None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:54.683099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1a55e-566d-4e99-9940-7d84f32c12b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:54.683754 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1a55e-566d-4e99-9940-7d84f32c12b6 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:54.684036 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1a55e-566d-4e99-9940-7d84f32c12b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:54.684309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c1a55e-566d-4e99-9940-7d84f32c12b6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:54.698230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 221/442] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:54.699009 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a41e6-1c02-44f3-80f9-db95201b4e56 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:54.740371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:54.754202 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c1a55e-566d-4e99-9940-7d84f32c12b6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:54.755703 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c1a55e-566d-4e99-9940-7d84f32c12b6 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:54.799984 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482a41e6-1c02-44f3-80f9-db95201b4e56 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:54.810974 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:54.834243 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae9157-a0c0-4a12-9665-c81ddc8b04fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:54.835144 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae9157-a0c0-4a12-9665-c81ddc8b04fe None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:54.835420 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae9157-a0c0-4a12-9665-c81ddc8b04fe None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:54.835684 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dae9157-a0c0-4a12-9665-c81ddc8b04fe None admin] PATH_INFO: `/v3/projects/4206726f0af54455a4cb03012949ffb5` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:54.836891 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae9157-a0c0-4a12-9665-c81ddc8b04fe None admin] RBAC: Authorizing `identity:get_project(project_id=4206726f0af54455a4cb03012949ffb5)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:54.838312 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dae9157-a0c0-4a12-9665-c81ddc8b04fe None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:54.840013 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 222/443] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:54 2022] GET /identity/v3/projects/4206726f0af54455a4cb03012949ffb5 => generated 302 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:54.840763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae9157-a0c0-4a12-9665-c81ddc8b04fe None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:54.862773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 222/444] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:53:54.863711 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1a55e-566d-4e99-9940-7d84f32c12b6 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:54.945877 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dae9157-a0c0-4a12-9665-c81ddc8b04fe None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:54.953019 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:54.964760 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c1a55e-566d-4e99-9940-7d84f32c12b6 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:54.980372 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9e2aa-82a4-48ff-acd5-44be0a39541a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:54.981182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9e2aa-82a4-48ff-acd5-44be0a39541a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:54.981454 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9e2aa-82a4-48ff-acd5-44be0a39541a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:54.981721 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed9e2aa-82a4-48ff-acd5-44be0a39541a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:54.991622 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed9e2aa-82a4-48ff-acd5-44be0a39541a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:53:54.997890 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f46f51-b1cb-4aae-8fbd-ebdf970198d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:54.998747 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f46f51-b1cb-4aae-8fbd-ebdf970198d6 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:54.999064 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f46f51-b1cb-4aae-8fbd-ebdf970198d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:54.999161 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-0ed9e2aa-82a4-48ff-acd5-44be0a39541a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:53:54.999161 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:54.999161 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:54.999161 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:54.999161 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:54.999161 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:54.999161 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:54.999161 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:54.999161 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:54.999161 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:54.999161 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:54.999161 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:54.999161 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:53:54.999161 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:53:54.999161 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:53:54.999161 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:53:55.000228 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:53:55.000228 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:53:55.000228 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:53:55.000228 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:53:55.000228 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:55.000228 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:55.000228 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:55.000228 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:55.000228 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:55.000228 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:55.000228 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:55.000228 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:55.000228 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:55.000228 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:55.000228 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:55.000228 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:55.001162 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:55.001162 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:55.001162 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:55.001162 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:55.001162 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:53:55.001162 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:53:55.001162 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:53:55.001162 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:53:55.001162 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:53:55.001162 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:53:55.001162 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:53:55.001162 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:55.001834 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f46f51-b1cb-4aae-8fbd-ebdf970198d6 None admin] PATH_INFO: `/v3/groups/df89dd397b0e496fad775ebf220818d2` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:55.001909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 223/445] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:53:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:55.001909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9e2aa-82a4-48ff-acd5-44be0a39541a None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:55.002787 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f46f51-b1cb-4aae-8fbd-ebdf970198d6 None admin] RBAC: Authorizing `identity:get_group(group_id=df89dd397b0e496fad775ebf220818d2)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:55.004339 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f46f51-b1cb-4aae-8fbd-ebdf970198d6 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:55.010829 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 223/446] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 23 16:53:54 2022] GET /identity/v3/groups/df89dd397b0e496fad775ebf220818d2 => generated 226 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:55.010829 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f46f51-b1cb-4aae-8fbd-ebdf970198d6 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:55.026250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:55.100418 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed9e2aa-82a4-48ff-acd5-44be0a39541a None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:55.107878 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f46f51-b1cb-4aae-8fbd-ebdf970198d6 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:55.134106 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08acef4-ead3-4fbe-a1e0-fb43491190c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:55.135020 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08acef4-ead3-4fbe-a1e0-fb43491190c7 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:55.135290 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08acef4-ead3-4fbe-a1e0-fb43491190c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:55.135598 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08acef4-ead3-4fbe-a1e0-fb43491190c7 None admin] PATH_INFO: `/v3/projects/4206726f0af54455a4cb03012949ffb5` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:55.136737 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08acef4-ead3-4fbe-a1e0-fb43491190c7 None admin] RBAC: Authorizing `identity:get_project(project_id=4206726f0af54455a4cb03012949ffb5)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:55.138205 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08acef4-ead3-4fbe-a1e0-fb43491190c7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:55.139888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 224/447] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:55 2022] GET /identity/v3/projects/4206726f0af54455a4cb03012949ffb5 => generated 302 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:55.144394 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08acef4-ead3-4fbe-a1e0-fb43491190c7 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:55.148825 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8757b1-704a-4a0c-a04a-cc0f933a2927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:55.149931 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8757b1-704a-4a0c-a04a-cc0f933a2927 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:55.150216 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8757b1-704a-4a0c-a04a-cc0f933a2927 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:55.150493 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8757b1-704a-4a0c-a04a-cc0f933a2927 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:55.151633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8757b1-704a-4a0c-a04a-cc0f933a2927 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:55.151953 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8757b1-704a-4a0c-a04a-cc0f933a2927 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:55.153628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8757b1-704a-4a0c-a04a-cc0f933a2927 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:55.162394 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 224/448] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:53:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:55.163212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8757b1-704a-4a0c-a04a-cc0f933a2927 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:55.178370 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:55.245409 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08acef4-ead3-4fbe-a1e0-fb43491190c7 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:55.245633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:55.266414 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8757b1-704a-4a0c-a04a-cc0f933a2927 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:55.278595 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2572013-262b-416d-b76a-57a002747e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:55.279277 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2572013-262b-416d-b76a-57a002747e9c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:55.279559 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2572013-262b-416d-b76a-57a002747e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:55.279825 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2572013-262b-416d-b76a-57a002747e9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:55.280644 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2572013-262b-416d-b76a-57a002747e9c None admin] RBAC: Adding query filter params (group.id=df89dd397b0e496fad775ebf220818d2, role.id=72cc26a1a66c4fc183b39fad4cb38431, scope.project.id=4206726f0af54455a4cb03012949ffb5) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:55.281004 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2572013-262b-416d-b76a-57a002747e9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:55.282422 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2572013-262b-416d-b76a-57a002747e9c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:55.288524 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 225/449] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 23 16:53:55 2022] GET /identity/v3/role_assignments?group.id=df89dd397b0e496fad775ebf220818d2&scope.project.id=4206726f0af54455a4cb03012949ffb5&role.id=72cc26a1a66c4fc183b39fad4cb38431 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:55.289291 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2572013-262b-416d-b76a-57a002747e9c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:55.302724 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de77c1f-b3c8-48c9-bc96-3df8886cab1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:55.303679 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de77c1f-b3c8-48c9-bc96-3df8886cab1c None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:55.303924 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de77c1f-b3c8-48c9-bc96-3df8886cab1c None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:55.304219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de77c1f-b3c8-48c9-bc96-3df8886cab1c None admin] PATH_INFO: `/v3/groups/df89dd397b0e496fad775ebf220818d2` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:55.305539 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de77c1f-b3c8-48c9-bc96-3df8886cab1c None admin] RBAC: Authorizing `identity:get_group(group_id=df89dd397b0e496fad775ebf220818d2)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:55.307061 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de77c1f-b3c8-48c9-bc96-3df8886cab1c None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:55.308800 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 225/450] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 23 16:53:55 2022] GET /identity/v3/groups/df89dd397b0e496fad775ebf220818d2 => generated 226 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:55.314883 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de77c1f-b3c8-48c9-bc96-3df8886cab1c None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:55.414964 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:55.419737 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2572013-262b-416d-b76a-57a002747e9c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:55.438595 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de77c1f-b3c8-48c9-bc96-3df8886cab1c None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:55.471371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11cba19-451e-4358-9d22-7399a6c4c612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:55.472023 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cba19-451e-4358-9d22-7399a6c4c612 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:55.472310 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cba19-451e-4358-9d22-7399a6c4c612 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:55.472582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11cba19-451e-4358-9d22-7399a6c4c612 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:55.478387 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce546e2e-2ac2-4542-b5f1-36cda0b6723d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:55.479084 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce546e2e-2ac2-4542-b5f1-36cda0b6723d None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:55.479363 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce546e2e-2ac2-4542-b5f1-36cda0b6723d None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:55.479631 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce546e2e-2ac2-4542-b5f1-36cda0b6723d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:55.480462 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce546e2e-2ac2-4542-b5f1-36cda0b6723d None admin] RBAC: Adding query filter params (group.id=df89dd397b0e496fad775ebf220818d2, role.id=6c30aca17fad4c4fa40b9461c5bda291, scope.project.id=4206726f0af54455a4cb03012949ffb5) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:55.480770 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce546e2e-2ac2-4542-b5f1-36cda0b6723d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:55.482283 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce546e2e-2ac2-4542-b5f1-36cda0b6723d None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:55.488780 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 226/451] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 23 16:53:55 2022] GET /identity/v3/role_assignments?group.id=df89dd397b0e496fad775ebf220818d2&scope.project.id=4206726f0af54455a4cb03012949ffb5&role.id=6c30aca17fad4c4fa40b9461c5bda291 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:55.564565 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11cba19-451e-4358-9d22-7399a6c4c612 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:55.565994 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e11cba19-451e-4358-9d22-7399a6c4c612 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:55.679968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:55.687254 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 226/452] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:53:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:53:55.689807 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce546e2e-2ac2-4542-b5f1-36cda0b6723d None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:55.710749 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:55.802350 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce546e2e-2ac2-4542-b5f1-36cda0b6723d None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:55.834889 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87cc49-8629-4847-b42e-e975b9fea6c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:55.835762 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87cc49-8629-4847-b42e-e975b9fea6c1 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:55.836046 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87cc49-8629-4847-b42e-e975b9fea6c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:55.836345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc87cc49-8629-4847-b42e-e975b9fea6c1 None admin] PATH_INFO: `/v3/projects/841c10778a1544ddb64ccc3d96fecf42` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:55.837525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc87cc49-8629-4847-b42e-e975b9fea6c1 None admin] RBAC: Authorizing `identity:get_project(project_id=841c10778a1544ddb64ccc3d96fecf42)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:55.838986 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc87cc49-8629-4847-b42e-e975b9fea6c1 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:55.840621 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 227/453] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:53:55 2022] GET /identity/v3/projects/841c10778a1544ddb64ccc3d96fecf42 => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:55.846162 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11cba19-451e-4358-9d22-7399a6c4c612 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:55.944245 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11cba19-451e-4358-9d22-7399a6c4c612 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:55.979674 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d10818-665d-4494-8e51-fc6b9182bd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:55.980520 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d10818-665d-4494-8e51-fc6b9182bd2c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:55.980792 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d10818-665d-4494-8e51-fc6b9182bd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:55.981077 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d10818-665d-4494-8e51-fc6b9182bd2c None admin] PATH_INFO: `/v3/groups/d79faf80a5004c57b2ced96c4bc51819` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:55.982337 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d10818-665d-4494-8e51-fc6b9182bd2c None admin] RBAC: Authorizing `identity:get_group(group_id=d79faf80a5004c57b2ced96c4bc51819)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:55.983824 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d10818-665d-4494-8e51-fc6b9182bd2c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:55.985433 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 227/454] 10.222.0.13 () {56 vars in 1180 bytes} [Mon May 23 16:53:55 2022] GET /identity/v3/groups/d79faf80a5004c57b2ced96c4bc51819 => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:55.994455 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87cc49-8629-4847-b42e-e975b9fea6c1 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:56.082249 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:56.107845 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc87cc49-8629-4847-b42e-e975b9fea6c1 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:56.143481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea898c4-b43e-43b4-b545-fa9ae8c1b5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:56.144094 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea898c4-b43e-43b4-b545-fa9ae8c1b5aa None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:56.144370 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea898c4-b43e-43b4-b545-fa9ae8c1b5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:56.144650 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea898c4-b43e-43b4-b545-fa9ae8c1b5aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:56.145567 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea898c4-b43e-43b4-b545-fa9ae8c1b5aa None admin] RBAC: Adding query filter params (group.id=d79faf80a5004c57b2ced96c4bc51819, role.id=17bbe7cc1c3641509de5e9dca8534102, scope.project.id=841c10778a1544ddb64ccc3d96fecf42) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:56.146120 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea898c4-b43e-43b4-b545-fa9ae8c1b5aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:56.147605 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea898c4-b43e-43b4-b545-fa9ae8c1b5aa None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:56.153406 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 228/455] 10.222.0.13 () {56 vars in 1376 bytes} [Mon May 23 16:53:55 2022] GET /identity/v3/role_assignments?group.id=d79faf80a5004c57b2ced96c4bc51819&scope.project.id=841c10778a1544ddb64ccc3d96fecf42&role.id=17bbe7cc1c3641509de5e9dca8534102 => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:58.698494 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957a60c-5350-4b41-a6e7-de00353806db None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:58.699549 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957a60c-5350-4b41-a6e7-de00353806db None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:58.700034 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e957a60c-5350-4b41-a6e7-de00353806db None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:58.701834 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 228/456] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:58.707717 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19658d-655f-40e8-8606-95d64d579334 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:58.708561 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19658d-655f-40e8-8606-95d64d579334 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:58.709051 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c19658d-655f-40e8-8606-95d64d579334 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:58.732057 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-6c19658d-655f-40e8-8606-95d64d579334 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:58.733229 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-6c19658d-655f-40e8-8606-95d64d579334 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:58.813464 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 229/457] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 16:53:58.852081 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957a60c-5350-4b41-a6e7-de00353806db None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:58.954410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:59.003685 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e957a60c-5350-4b41-a6e7-de00353806db None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:59.049194 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6e7a6-d8b3-48b6-a626-dabe6ba36900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:59.050019 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6e7a6-d8b3-48b6-a626-dabe6ba36900 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:59.050299 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6e7a6-d8b3-48b6-a626-dabe6ba36900 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:59.050571 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb6e7a6-d8b3-48b6-a626-dabe6ba36900 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:59.069051 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6e7a6-d8b3-48b6-a626-dabe6ba36900 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:59.070566 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb6e7a6-d8b3-48b6-a626-dabe6ba36900 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:59.099564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-bfb6e7a6-d8b3-48b6-a626-dabe6ba36900 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:53:59.099564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:59.099564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:59.099564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:59.099564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:59.099564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:59.099564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:59.099564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:59.099564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:59.099564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:59.099564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:59.099564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:59.099564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:53:59.099564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:53:59.099564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:53:59.100540 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:53:59.100540 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:59.100540 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:59.100540 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:59.100540 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:59.100540 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:59.100540 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:59.100540 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:59.100540 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:59.100540 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:59.100540 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:59.100540 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:59.100540 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:59.100540 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:59.100540 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:59.100540 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:59.100540 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:59.101363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:53:59.101363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:53:59.101363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:53:59.101363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:59.101363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 229/458] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:53:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:59.111074 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19658d-655f-40e8-8606-95d64d579334 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:59.235383 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:59.264964 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c19658d-655f-40e8-8606-95d64d579334 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:59.310692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae65af8b-90f4-498d-96cf-794c72e0b32d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:59.311412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65af8b-90f4-498d-96cf-794c72e0b32d None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:59.311694 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65af8b-90f4-498d-96cf-794c72e0b32d None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:59.311967 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae65af8b-90f4-498d-96cf-794c72e0b32d None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:59.312712 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae65af8b-90f4-498d-96cf-794c72e0b32d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:59.313018 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae65af8b-90f4-498d-96cf-794c72e0b32d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:59.314423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae65af8b-90f4-498d-96cf-794c72e0b32d None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:59.332550 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 230/459] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:53:59.338665 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6e7a6-d8b3-48b6-a626-dabe6ba36900 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:59.369440 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b67f04-4f7a-4ff8-8864-8ad7649b96d4 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:59.369742 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b67f04-4f7a-4ff8-8864-8ad7649b96d4 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:59.369991 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b67f04-4f7a-4ff8-8864-8ad7649b96d4 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:59.371664 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 231/460] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:53:59.377460 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4abdd-3e79-4bd4-be07-a263703b6308 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:59.377745 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4abdd-3e79-4bd4-be07-a263703b6308 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:59.378168 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d4abdd-3e79-4bd4-be07-a263703b6308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:59.405418 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-04d4abdd-3e79-4bd4-be07-a263703b6308 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:59.406702 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-04d4abdd-3e79-4bd4-be07-a263703b6308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:59.466030 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb6e7a6-d8b3-48b6-a626-dabe6ba36900 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:59.479126 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 232/461] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 16:53:59.480548 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a40b95-d575-4f60-b55a-f126b3bcbd5f None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:59.486465 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a40b95-d575-4f60-b55a-f126b3bcbd5f None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:59.486465 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a40b95-d575-4f60-b55a-f126b3bcbd5f None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:59.486465 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 233/462] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:59.488007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c8700-9416-4978-9e39-e3edf6b4fdb1 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:59.488286 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c8700-9416-4978-9e39-e3edf6b4fdb1 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:59.488558 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c8700-9416-4978-9e39-e3edf6b4fdb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:59.491415 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:59.499854 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44be7f-dcac-4e30-8b22-7f16856ff401 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:59.500533 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44be7f-dcac-4e30-8b22-7f16856ff401 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:59.500815 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44be7f-dcac-4e30-8b22-7f16856ff401 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:59.501091 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a44be7f-dcac-4e30-8b22-7f16856ff401 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:59.501897 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a44be7f-dcac-4e30-8b22-7f16856ff401 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:59.503407 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a44be7f-dcac-4e30-8b22-7f16856ff401 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:59.507151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-704c8700-9416-4978-9e39-e3edf6b4fdb1 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:59.508879 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-704c8700-9416-4978-9e39-e3edf6b4fdb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:59.529024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 230/463] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 23 16:53:59 2022] POST /identity/v3/users => generated 246 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:53:59.530008 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44be7f-dcac-4e30-8b22-7f16856ff401 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:59.576498 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:59.591051 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 234/464] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:59.592609 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760ab86-bed5-4a59-bcd5-3ac2208880e7 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:59.592896 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760ab86-bed5-4a59-bcd5-3ac2208880e7 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:59.593171 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b760ab86-bed5-4a59-bcd5-3ac2208880e7 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:59.595862 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 235/465] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:53:59.600698 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afa27c-9402-4f86-bd9d-7d10751c979a None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:59.601013 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afa27c-9402-4f86-bd9d-7d10751c979a None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:59.601251 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58afa27c-9402-4f86-bd9d-7d10751c979a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:59.627741 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-58afa27c-9402-4f86-bd9d-7d10751c979a None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:53:59.628933 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-58afa27c-9402-4f86-bd9d-7d10751c979a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:53:59.653876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a44be7f-dcac-4e30-8b22-7f16856ff401 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:59.694144 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d757320-8f85-4894-9842-2f2815a26d68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:59.694988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d757320-8f85-4894-9842-2f2815a26d68 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:59.695276 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d757320-8f85-4894-9842-2f2815a26d68 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:59.695650 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d757320-8f85-4894-9842-2f2815a26d68 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:59.709473 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d757320-8f85-4894-9842-2f2815a26d68 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:59.711025 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d757320-8f85-4894-9842-2f2815a26d68 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:59.711276 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 236/466] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:53:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:53:59.712342 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afa27c-9402-4f86-bd9d-7d10751c979a None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:59.730174 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-4d757320-8f85-4894-9842-2f2815a26d68 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:53:59.730174 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:59.730174 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:59.730174 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:59.730174 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:59.730174 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:59.730174 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:59.730174 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:59.730174 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:59.730174 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:59.730174 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:59.730174 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:59.730174 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:53:59.730174 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:53:59.730174 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:53:59.732513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:53:59.732513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:59.732513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:59.732513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:59.732513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:59.732513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:59.732513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:59.732513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:59.732513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:53:59.732513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:59.732513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:59.732513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:59.732513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:59.732513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:59.732513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:59.732513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:59.732513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:59.734577 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:53:59.734577 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:53:59.734577 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:53:59.734577 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:53:59.734577 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 231/467] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:59.739517 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d757320-8f85-4894-9842-2f2815a26d68 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:59.769435 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:59.835775 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58afa27c-9402-4f86-bd9d-7d10751c979a None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:59.863175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d757320-8f85-4894-9842-2f2815a26d68 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:53:59.868577 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:53:59.885480 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad6c9c-d2b9-422f-96da-56a84ee8733d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:59.886725 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad6c9c-d2b9-422f-96da-56a84ee8733d None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:59.887813 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad6c9c-d2b9-422f-96da-56a84ee8733d None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:59.888311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ad6c9c-d2b9-422f-96da-56a84ee8733d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:59.903290 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1fbe9-e7c0-40ad-b952-bd5b579f64cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:53:59.903928 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1fbe9-e7c0-40ad-b952-bd5b579f64cd None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:53:59.904203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1fbe9-e7c0-40ad-b952-bd5b579f64cd None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:53:59.904472 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a1fbe9-e7c0-40ad-b952-bd5b579f64cd None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:53:59.905194 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a1fbe9-e7c0-40ad-b952-bd5b579f64cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:53:59.905607 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a1fbe9-e7c0-40ad-b952-bd5b579f64cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:59.906607 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad6c9c-d2b9-422f-96da-56a84ee8733d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:53:59.906939 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a1fbe9-e7c0-40ad-b952-bd5b579f64cd None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:59.908805 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5ad6c9c-d2b9-422f-96da-56a84ee8733d None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:53:59.920103 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 232/468] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:53:59.925184 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1fbe9-e7c0-40ad-b952-bd5b579f64cd None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:53:59.934352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-c5ad6c9c-d2b9-422f-96da-56a84ee8733d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:53:59.934352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:53:59.934352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:53:59.934352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:53:59.934352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:53:59.934352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:53:59.934352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:53:59.934352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:53:59.934352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:53:59.934352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:53:59.934352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:53:59.934352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:53:59.934352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:53:59.934352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:53:59.934352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:53:59.935291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:53:59.935291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:53:59.935291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:53:59.935291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:53:59.935291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:53:59.935291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:53:59.935291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:53:59.935291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:53:59.935291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:53:59.935291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:53:59.935291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:53:59.935291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:53:59.935291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:53:59.935291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:53:59.935291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:53:59.935291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:53:59.935291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:53:59.936132 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:53:59.936132 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:53:59.936132 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:53:59.936132 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:53:59.940383 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 237/469] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:53:59.941239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad6c9c-d2b9-422f-96da-56a84ee8733d None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:00.048803 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:00.059829 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ad6c9c-d2b9-422f-96da-56a84ee8733d None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:00.065049 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a1fbe9-e7c0-40ad-b952-bd5b579f64cd None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:00.102019 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3fada-d6dc-4e77-b500-37668108a13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:00.102825 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3fada-d6dc-4e77-b500-37668108a13c None admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:00.103276 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3fada-d6dc-4e77-b500-37668108a13c None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:00.103629 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3fada-d6dc-4e77-b500-37668108a13c None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:00.104396 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3fada-d6dc-4e77-b500-37668108a13c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:00.105921 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3fada-d6dc-4e77-b500-37668108a13c None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:00.108298 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3191551-cbaa-4e00-bbc2-a99b1325df67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:00.109318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3191551-cbaa-4e00-bbc2-a99b1325df67 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:00.109782 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3191551-cbaa-4e00-bbc2-a99b1325df67 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:00.110224 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3191551-cbaa-4e00-bbc2-a99b1325df67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:00.123717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3191551-cbaa-4e00-bbc2-a99b1325df67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:00.125571 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3191551-cbaa-4e00-bbc2-a99b1325df67 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:00.134474 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 238/470] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 23 16:53:59 2022] POST /identity/v3/users => generated 248 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:54:00.135893 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3fada-d6dc-4e77-b500-37668108a13c None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:00.145790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-a3191551-cbaa-4e00-bbc2-a99b1325df67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:00.145790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:00.145790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:00.145790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:00.145790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:00.145790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:00.145790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:00.145790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:00.145790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:00.145790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:00.145790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:00.145790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:00.145790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:00.145790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:00.145790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:00.146687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:00.146687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:00.146687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:00.146687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:00.146687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:00.146687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:00.146687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:00.146687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:00.146687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:00.146687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:00.146687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:00.146687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:00.146687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:00.146687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:00.146687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:00.146687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:00.146687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:00.147582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:00.147582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:00.147582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:00.147582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:00.149575 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 233/471] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:00.160071 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3191551-cbaa-4e00-bbc2-a99b1325df67 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:00.200988 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:00.285249 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3fada-d6dc-4e77-b500-37668108a13c None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:00.307859 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3191551-cbaa-4e00-bbc2-a99b1325df67 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:00.328107 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f67218-fec4-4db9-9391-c577b1c50043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:00.328791 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f67218-fec4-4db9-9391-c577b1c50043 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:00.329072 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f67218-fec4-4db9-9391-c577b1c50043 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:00.329346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f67218-fec4-4db9-9391-c577b1c50043 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:00.330085 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f67218-fec4-4db9-9391-c577b1c50043 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:00.330392 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f67218-fec4-4db9-9391-c577b1c50043 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:00.331819 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f67218-fec4-4db9-9391-c577b1c50043 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:00.347038 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 239/472] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:00.348298 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f6be8-3c65-4251-8304-ff5ac56febfe None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:00.348587 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f6be8-3c65-4251-8304-ff5ac56febfe None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:00.348860 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50f6be8-3c65-4251-8304-ff5ac56febfe None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:00.350369 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 240/473] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:00.357685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f6be8-3c65-4251-8304-ff5ac56febfe None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:00.369136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a6a395-16be-4a7b-aa41-8f6b343130d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:00.369478 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6a395-16be-4a7b-aa41-8f6b343130d3 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:00.369756 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6a395-16be-4a7b-aa41-8f6b343130d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:00.370405 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a6a395-16be-4a7b-aa41-8f6b343130d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:00.371172 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a6a395-16be-4a7b-aa41-8f6b343130d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:00.371785 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a6a395-16be-4a7b-aa41-8f6b343130d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:00.373193 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a6a395-16be-4a7b-aa41-8f6b343130d3 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:00.381530 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:00.387964 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 234/474] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:00.392771 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9e334-3f07-4f5f-ad40-a3eba266a89a None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:00.393412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9e334-3f07-4f5f-ad40-a3eba266a89a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:00.394103 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf9e334-3f07-4f5f-ad40-a3eba266a89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:00.416230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-6cf9e334-3f07-4f5f-ad40-a3eba266a89a None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:00.417469 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-6cf9e334-3f07-4f5f-ad40-a3eba266a89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:00.470397 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:00.486945 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50f6be8-3c65-4251-8304-ff5ac56febfe None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:00.503117 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 235/475] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:54:00.504286 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9e334-3f07-4f5f-ad40-a3eba266a89a None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:00.536790 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d6e61-db07-4184-ad21-3049f10c7016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:00.537575 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d6e61-db07-4184-ad21-3049f10c7016 None admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:00.537851 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d6e61-db07-4184-ad21-3049f10c7016 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:00.538122 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9d6e61-db07-4184-ad21-3049f10c7016 None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:00.538826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d6e61-db07-4184-ad21-3049f10c7016 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:00.540345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9d6e61-db07-4184-ad21-3049f10c7016 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:00.567068 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 241/476] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 23 16:54:00 2022] POST /identity/v3/users => generated 249 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 16:54:00.568104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d6e61-db07-4184-ad21-3049f10c7016 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:00.694403 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf9e334-3f07-4f5f-ad40-a3eba266a89a None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:00.749149 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:00.761152 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fcca6-6645-4f80-b96f-f5082587c30c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:00.762237 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fcca6-6645-4f80-b96f-f5082587c30c None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:00.762733 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fcca6-6645-4f80-b96f-f5082587c30c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:00.763240 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995fcca6-6645-4f80-b96f-f5082587c30c None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:00.764826 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995fcca6-6645-4f80-b96f-f5082587c30c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:00.766547 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995fcca6-6645-4f80-b96f-f5082587c30c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:00.791551 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 236/477] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 23 16:54:00 2022] POST /identity/v3/users => generated 248 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:54:00.804113 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9d6e61-db07-4184-ad21-3049f10c7016 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:00.861143 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d3373-4f0b-4a73-8893-a33dc07f9b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:00.861841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d3373-4f0b-4a73-8893-a33dc07f9b16 None admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:00.862123 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d3373-4f0b-4a73-8893-a33dc07f9b16 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:00.862400 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf4d3373-4f0b-4a73-8893-a33dc07f9b16 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:00.863219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d3373-4f0b-4a73-8893-a33dc07f9b16 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:00.865484 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf4d3373-4f0b-4a73-8893-a33dc07f9b16 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:00.875328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.sql.core [None req-bf4d3373-4f0b-4a73-8893-a33dc07f9b16 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 23 16:54:00.875328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 23 16:54:00.875328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [parameters: {'id': 'c089aaa9f5584166a16d60f4873dc079', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 23 16:54:00.875328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80465) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 23 16:54:00.893663 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-bf4d3373-4f0b-4a73-8893-a33dc07f9b16 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 23 16:54:00.893663 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:00.893663 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 16:54:00.893663 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 16:54:00.893663 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 16:54:00.893663 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 16:54:00.893663 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 16:54:00.893663 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application result = self._query(query) May 23 16:54:00.893663 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 16:54:00.893663 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application conn.query(q) May 23 16:54:00.893663 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 16:54:00.893663 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 16:54:00.893663 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 16:54:00.893663 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application result.read() May 23 16:54:00.893663 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 16:54:00.894640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 16:54:00.894640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 16:54:00.894640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 16:54:00.894640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 16:54:00.894640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 16:54:00.894640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 16:54:00.894640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 16:54:00.894640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 23 16:54:00.894640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:00.894640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 23 16:54:00.894640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:00.894640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:00.894640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 23 16:54:00.894640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 23 16:54:00.894640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 23 16:54:00.894640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return ref.to_dict() May 23 16:54:00.894640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 16:54:00.894640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application next(self.gen) May 23 16:54:00.895761 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 23 16:54:00.895761 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application yield resource May 23 16:54:00.895761 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 16:54:00.895761 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application next(self.gen) May 23 16:54:00.895761 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 23 16:54:00.895761 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self.session.rollback() May 23 16:54:00.895761 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 23 16:54:00.895761 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self.force_reraise() May 23 16:54:00.895761 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 23 16:54:00.895761 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise self.value May 23 16:54:00.895761 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 23 16:54:00.895761 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 23 16:54:00.895761 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 23 16:54:00.895761 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application session.commit() May 23 16:54:00.895761 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 23 16:54:00.895761 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 23 16:54:00.895761 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 23 16:54:00.896823 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self._prepare_impl() May 23 16:54:00.896823 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 23 16:54:00.896823 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self.session.flush() May 23 16:54:00.896823 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 23 16:54:00.896823 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self._flush(objects) May 23 16:54:00.896823 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 23 16:54:00.896823 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 23 16:54:00.896823 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 23 16:54:00.896823 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application compat.raise_( May 23 16:54:00.896823 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 16:54:00.896823 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception May 23 16:54:00.896823 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 23 16:54:00.896823 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application flush_context.execute() May 23 16:54:00.896823 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 23 16:54:00.896823 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rec.execute(self) May 23 16:54:00.896823 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 23 16:54:00.896823 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 23 16:54:00.897748 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 23 16:54:00.897748 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application _emit_insert_statements( May 23 16:54:00.897748 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 23 16:54:00.897748 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application c = connection._execute_20( May 23 16:54:00.897748 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 23 16:54:00.897748 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 23 16:54:00.897748 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 23 16:54:00.897748 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 23 16:54:00.897748 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 23 16:54:00.897748 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ret = self._execute_context( May 23 16:54:00.897748 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 23 16:54:00.897748 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 23 16:54:00.897748 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 23 16:54:00.897748 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 23 16:54:00.897748 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 16:54:00.897748 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception May 23 16:54:00.898562 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 16:54:00.898562 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 16:54:00.898562 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 16:54:00.898562 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 16:54:00.898562 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 16:54:00.898562 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application result = self._query(query) May 23 16:54:00.898562 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 16:54:00.898562 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application conn.query(q) May 23 16:54:00.898562 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 16:54:00.898562 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 16:54:00.898562 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 16:54:00.898562 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application result.read() May 23 16:54:00.898562 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 16:54:00.898562 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 16:54:00.898562 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 16:54:00.898562 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 16:54:00.899408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 16:54:00.899408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 16:54:00.899408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 16:54:00.899408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 16:54:00.899408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 23 16:54:00.899408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 23 16:54:00.899408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [parameters: {'id': 'c089aaa9f5584166a16d60f4873dc079', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 23 16:54:00.899408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 23 16:54:00.899408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:00.899408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:54:00.899408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:00.899408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:00.899408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:00.899408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:00.899408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:00.899408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:00.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:00.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:00.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:00.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:00.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:00.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:00.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 23 16:54:00.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 23 16:54:00.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:00.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:00.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 23 16:54:00.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 23 16:54:00.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 23 16:54:00.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 23 16:54:00.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 23 16:54:00.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:00.901094 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 242/478] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 23 16:54:00 2022] POST /identity/v3/roles => generated 144 bytes in 328 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:00.910439 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fcca6-6645-4f80-b96f-f5082587c30c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:01.065030 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995fcca6-6645-4f80-b96f-f5082587c30c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:01.099707 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd6f474-3d04-4455-aa3f-524c72eb1d0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:01.100712 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6f474-3d04-4455-aa3f-524c72eb1d0b None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:01.101092 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6f474-3d04-4455-aa3f-524c72eb1d0b None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:01.101463 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd6f474-3d04-4455-aa3f-524c72eb1d0b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:01.112400 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd6f474-3d04-4455-aa3f-524c72eb1d0b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:01.120914 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-8cd6f474-3d04-4455-aa3f-524c72eb1d0b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:54:01.120914 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:01.120914 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:01.120914 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:01.120914 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:01.120914 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:01.120914 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:01.120914 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:01.120914 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:01.120914 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:01.120914 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:01.120914 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:01.120914 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:01.120914 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:01.120914 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:01.120914 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:01.122002 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:01.122002 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:01.122002 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:01.122002 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:01.122002 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:01.122002 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:01.122002 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:01.122002 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:01.122002 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:01.122002 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:01.122002 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:01.122002 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:01.122002 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:01.122002 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:01.122002 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:01.122002 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:01.122988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:01.122988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:01.122988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:01.122988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:01.122988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:01.122988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:01.122988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:01.122988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:01.122988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:01.122988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:01.122988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 16:54:01.122988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:01.125114 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 237/479] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 23 16:54:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:01.130862 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d3373-4f0b-4a73-8893-a33dc07f9b16 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:01.150053 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:01.247803 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf4d3373-4f0b-4a73-8893-a33dc07f9b16 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:01.287332 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea74b6b7-cee2-42fb-817a-c383114cf725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:01.288016 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74b6b7-cee2-42fb-817a-c383114cf725 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:01.288296 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74b6b7-cee2-42fb-817a-c383114cf725 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:01.288568 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea74b6b7-cee2-42fb-817a-c383114cf725 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:01.289326 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74b6b7-cee2-42fb-817a-c383114cf725 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:01.289632 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74b6b7-cee2-42fb-817a-c383114cf725 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:01.291042 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea74b6b7-cee2-42fb-817a-c383114cf725 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:01.300261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 243/480] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:01 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:02.267518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765313e-97b0-4636-bbea-5f5a808d37fc None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:02.268037 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765313e-97b0-4636-bbea-5f5a808d37fc None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:02.268037 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0765313e-97b0-4636-bbea-5f5a808d37fc None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:02.269683 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 238/481] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:02.280485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6e730-72c7-4605-9bf6-221fa86ee8d9 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:02.280928 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6e730-72c7-4605-9bf6-221fa86ee8d9 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:02.281123 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e6e730-72c7-4605-9bf6-221fa86ee8d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:02.303901 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-e1e6e730-72c7-4605-9bf6-221fa86ee8d9 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:02.305098 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-e1e6e730-72c7-4605-9bf6-221fa86ee8d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:02.385846 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 244/482] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:54:02.424406 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765313e-97b0-4636-bbea-5f5a808d37fc None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:02.494259 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:02.578330 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0765313e-97b0-4636-bbea-5f5a808d37fc None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:02.613164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c15506d-9d18-4745-ab85-3d3041f7a720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:02.614010 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15506d-9d18-4745-ab85-3d3041f7a720 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:02.614299 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15506d-9d18-4745-ab85-3d3041f7a720 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:02.614556 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c15506d-9d18-4745-ab85-3d3041f7a720 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:02.624740 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c15506d-9d18-4745-ab85-3d3041f7a720 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:02.632902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-8c15506d-9d18-4745-ab85-3d3041f7a720 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:02.632902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:02.632902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:02.632902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:02.632902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:02.632902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:02.632902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:02.632902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:02.632902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:02.632902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:02.632902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:02.632902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:02.632902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:02.632902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:02.632902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:02.632902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:02.634141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:02.634141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:02.634141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:02.634141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:02.634141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:02.634141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:02.634141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:02.634141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:02.634141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:02.634141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:02.634141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:02.634141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:02.634141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:02.634141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:02.634141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:02.634141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:02.634953 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:02.634953 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:02.634953 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:02.634953 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:02.634953 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:02.634953 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:02.634953 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:02.634953 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:02.634953 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:02.634953 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:02.634953 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:02.634953 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:02.634953 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 239/483] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 23 16:54:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:02.640271 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e6e730-72c7-4605-9bf6-221fa86ee8d9 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:02.643597 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:02.749920 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622b29a-fcfc-45c9-89f1-eb29aac9d33d None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:02.750276 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622b29a-fcfc-45c9-89f1-eb29aac9d33d None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:02.750465 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622b29a-fcfc-45c9-89f1-eb29aac9d33d None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:02.755024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 240/484] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:02.760796 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae2c28-42c1-43d7-a431-c74b90ef4485 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:02.761322 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae2c28-42c1-43d7-a431-c74b90ef4485 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:02.761788 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adae2c28-42c1-43d7-a431-c74b90ef4485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:02.772751 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1e6e730-72c7-4605-9bf6-221fa86ee8d9 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:02.779917 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-adae2c28-42c1-43d7-a431-c74b90ef4485 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:02.781443 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-adae2c28-42c1-43d7-a431-c74b90ef4485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:02.820407 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7a513-7141-41b1-b387-fd72ff0164dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:02.821091 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7a513-7141-41b1-b387-fd72ff0164dc None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:02.821376 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7a513-7141-41b1-b387-fd72ff0164dc None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:02.821652 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7a513-7141-41b1-b387-fd72ff0164dc None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:02.822403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7a513-7141-41b1-b387-fd72ff0164dc None admin] RBAC: Adding query filter params (name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:02.822716 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7a513-7141-41b1-b387-fd72ff0164dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:02.824236 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b7a513-7141-41b1-b387-fd72ff0164dc None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:02.833364 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 245/485] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 23 16:54:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:02.839156 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7a513-7141-41b1-b387-fd72ff0164dc None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:02.917352 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 241/486] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 16:54:02.951306 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:02.956637 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae2c28-42c1-43d7-a431-c74b90ef4485 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:03.001806 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7a513-7141-41b1-b387-fd72ff0164dc None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:03.017120 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:03.075789 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c70a099-dedc-43d3-a61a-807243d3b654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:03.076600 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c70a099-dedc-43d3-a61a-807243d3b654 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:03.076878 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c70a099-dedc-43d3-a61a-807243d3b654 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:03.077150 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c70a099-dedc-43d3-a61a-807243d3b654 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:03.100343 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c70a099-dedc-43d3-a61a-807243d3b654 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:03.101835 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c70a099-dedc-43d3-a61a-807243d3b654 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:03.129134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-1c70a099-dedc-43d3-a61a-807243d3b654 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:03.129134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:03.129134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:03.129134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:03.129134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:03.129134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:03.129134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:03.129134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:03.129134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:03.129134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:03.129134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:03.129134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:03.129134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:03.129134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:03.129134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:03.130196 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:03.130196 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:03.130196 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:03.130196 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:03.130196 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:03.130196 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:03.130196 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:03.130196 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:03.130196 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:03.130196 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:03.130196 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:03.130196 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:03.130196 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:03.130196 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:03.130196 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:03.130196 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:03.130196 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:03.131063 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:03.131063 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:03.131063 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:03.131063 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:03.131315 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 246/487] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:54:03.138770 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c70a099-dedc-43d3-a61a-807243d3b654 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:03.157713 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adae2c28-42c1-43d7-a431-c74b90ef4485 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:03.250356 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c70a099-dedc-43d3-a61a-807243d3b654 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:03.252176 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af403e58-d4de-44e3-accb-5d1aa6a90992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:03.253090 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af403e58-d4de-44e3-accb-5d1aa6a90992 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:03.253377 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af403e58-d4de-44e3-accb-5d1aa6a90992 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:03.253650 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af403e58-d4de-44e3-accb-5d1aa6a90992 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:03.265518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af403e58-d4de-44e3-accb-5d1aa6a90992 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:03.270630 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:03.276019 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-af403e58-d4de-44e3-accb-5d1aa6a90992 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:03.276019 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:03.276019 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:03.276019 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:03.276019 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:03.276019 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:03.276019 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:03.276019 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:03.276019 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:03.276019 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:03.276019 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:03.276019 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:03.276019 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:03.276019 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:03.276019 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:03.276019 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:03.277323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:03.277323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:03.277323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:03.277323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:03.277323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:03.277323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:03.277323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:03.277323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:03.277323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:03.277323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:03.277323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:03.277323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:03.277323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:03.277323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:03.277323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:03.277323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:03.278136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:03.278136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:03.278136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:03.278136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:03.278136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:03.278136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:03.278136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:03.278136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:03.278136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:03.278136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:03.278136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:03.278136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:03.278136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 242/488] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 23 16:54:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:03.279261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca414c1-f7e5-4705-b3ad-46870440d5e3 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:03.279261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca414c1-f7e5-4705-b3ad-46870440d5e3 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:03.281210 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca414c1-f7e5-4705-b3ad-46870440d5e3 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:03.281210 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 243/489] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:03.286221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca414c1-f7e5-4705-b3ad-46870440d5e3 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:03.288720 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b5010f7-a067-44bb-b482-701ae96214a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:03.289733 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5010f7-a067-44bb-b482-701ae96214a8 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:03.290025 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5010f7-a067-44bb-b482-701ae96214a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:03.291116 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b5010f7-a067-44bb-b482-701ae96214a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:03.295418 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5010f7-a067-44bb-b482-701ae96214a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:03.295834 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5010f7-a067-44bb-b482-701ae96214a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:03.297853 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b5010f7-a067-44bb-b482-701ae96214a8 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:03.315270 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 247/490] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:03.316954 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cfe3d-f0a8-431e-80b0-20f969cdaa7d None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:03.317341 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cfe3d-f0a8-431e-80b0-20f969cdaa7d None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:03.317713 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953cfe3d-f0a8-431e-80b0-20f969cdaa7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:03.337681 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-953cfe3d-f0a8-431e-80b0-20f969cdaa7d None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:03.339127 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-953cfe3d-f0a8-431e-80b0-20f969cdaa7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:03.393962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:03.402619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca414c1-f7e5-4705-b3ad-46870440d5e3 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:03.415124 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 248/491] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:54:03.416006 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953cfe3d-f0a8-431e-80b0-20f969cdaa7d None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:03.460634 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f036ed-5faf-43fc-af2a-cfc0363089b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:03.461377 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f036ed-5faf-43fc-af2a-cfc0363089b9 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:03.461717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f036ed-5faf-43fc-af2a-cfc0363089b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:03.461995 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f036ed-5faf-43fc-af2a-cfc0363089b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:03.462756 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f036ed-5faf-43fc-af2a-cfc0363089b9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:03.466007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f036ed-5faf-43fc-af2a-cfc0363089b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:03.466007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f036ed-5faf-43fc-af2a-cfc0363089b9 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:03.487586 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 244/492] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 23 16:54:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:03.487586 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f036ed-5faf-43fc-af2a-cfc0363089b9 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:03.520863 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953cfe3d-f0a8-431e-80b0-20f969cdaa7d None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:03.530896 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:03.557928 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043380d-3506-406c-9167-9e3996b34832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:03.558934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043380d-3506-406c-9167-9e3996b34832 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:03.559293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043380d-3506-406c-9167-9e3996b34832 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:03.559726 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6043380d-3506-406c-9167-9e3996b34832 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:03.567540 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043380d-3506-406c-9167-9e3996b34832 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:03.569216 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6043380d-3506-406c-9167-9e3996b34832 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:03.586222 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-6043380d-3506-406c-9167-9e3996b34832 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 16:54:03.586222 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:03.586222 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:03.586222 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:03.586222 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:03.586222 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:03.586222 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:03.586222 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:03.586222 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:03.586222 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:03.586222 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:03.586222 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:03.586222 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:03.586222 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:03.586222 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:03.586222 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:03.587228 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:03.587228 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:03.587228 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:03.587228 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:03.587228 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:03.587228 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:03.587228 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:03.587228 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:03.587228 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:03.587228 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:03.587228 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:03.587228 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:03.587228 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:03.587228 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:03.587228 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:03.587228 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:03.587228 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:03.587228 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:03.588122 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:03.588122 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:03.588122 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:03.588122 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:03.588122 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:03.588122 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:03.588122 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:03.588122 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:03.588122 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 16:54:03.588122 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:03.588122 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 249/493] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 23 16:54:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:03.589175 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043380d-3506-406c-9167-9e3996b34832 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:03.601118 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f036ed-5faf-43fc-af2a-cfc0363089b9 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:03.639007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61626296-429b-47a2-8cff-c00655e62f6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:03.639829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61626296-429b-47a2-8cff-c00655e62f6c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:03.640110 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61626296-429b-47a2-8cff-c00655e62f6c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:03.640442 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61626296-429b-47a2-8cff-c00655e62f6c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:03.653734 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61626296-429b-47a2-8cff-c00655e62f6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:03.662367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-61626296-429b-47a2-8cff-c00655e62f6c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:03.662367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:03.662367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:03.662367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:03.662367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:03.662367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:03.662367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:03.662367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:03.662367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:03.662367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:03.662367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:03.662367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:03.662367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:03.662367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:03.662367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:03.662367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:03.664493 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:03.664493 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:03.664493 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:03.664493 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:03.664493 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:03.664493 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:03.664493 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:03.664493 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:03.664493 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:03.664493 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:03.664493 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:03.664493 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:03.664493 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:03.664493 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:03.664493 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:03.664493 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:03.666361 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:03.666361 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:03.666361 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:03.666361 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:03.666361 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:03.666361 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:03.666361 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:03.666361 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:03.666361 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:03.666361 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:03.666361 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:03.666361 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:03.673293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 245/494] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 23 16:54:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:03.673293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61626296-429b-47a2-8cff-c00655e62f6c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:03.691328 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:03.702446 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6043380d-3506-406c-9167-9e3996b34832 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:03.737338 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a3d3f-aae4-482f-999f-7fc6b84e9992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:03.738142 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a3d3f-aae4-482f-999f-7fc6b84e9992 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:03.738426 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a3d3f-aae4-482f-999f-7fc6b84e9992 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:03.738729 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39a3d3f-aae4-482f-999f-7fc6b84e9992 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:03.750025 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a3d3f-aae4-482f-999f-7fc6b84e9992 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:03.751531 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39a3d3f-aae4-482f-999f-7fc6b84e9992 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:03.769628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-e39a3d3f-aae4-482f-999f-7fc6b84e9992 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:03.769628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:03.769628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:03.769628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:03.769628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:03.769628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:03.769628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:03.769628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:03.769628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:03.769628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:03.769628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:03.769628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:03.769628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:03.769628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:03.769628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:03.770792 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:03.770792 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:03.770792 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:03.770792 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:03.770792 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:03.770792 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:03.770792 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:03.770792 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:03.770792 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:03.770792 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:03.770792 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:03.770792 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:03.770792 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:03.770792 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:03.770792 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:03.770792 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:03.770792 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:03.771744 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:03.771744 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:03.771744 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:03.771744 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:03.771744 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 250/495] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:03.772126 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a3d3f-aae4-482f-999f-7fc6b84e9992 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:03.788791 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61626296-429b-47a2-8cff-c00655e62f6c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:03.794354 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:03.828076 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958b186-0d16-414b-b113-8040bdfa934f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:03.829127 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958b186-0d16-414b-b113-8040bdfa934f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:03.829645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958b186-0d16-414b-b113-8040bdfa934f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:03.830059 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b958b186-0d16-414b-b113-8040bdfa934f None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:03.831065 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958b186-0d16-414b-b113-8040bdfa934f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:03.831520 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958b186-0d16-414b-b113-8040bdfa934f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:03.833391 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b958b186-0d16-414b-b113-8040bdfa934f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:03.874041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 246/496] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 23 16:54:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:03.875702 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958b186-0d16-414b-b113-8040bdfa934f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:03.878307 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39a3d3f-aae4-482f-999f-7fc6b84e9992 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:03.915927 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f15887-2226-4a6a-a120-25c6c82bc34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:03.916521 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f15887-2226-4a6a-a120-25c6c82bc34c None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:03.916804 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f15887-2226-4a6a-a120-25c6c82bc34c None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:03.917079 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f15887-2226-4a6a-a120-25c6c82bc34c None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:03.917837 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f15887-2226-4a6a-a120-25c6c82bc34c None admin] RBAC: Adding query filter params (name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:03.918149 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f15887-2226-4a6a-a120-25c6c82bc34c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:03.919695 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f15887-2226-4a6a-a120-25c6c82bc34c None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:03.932554 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 251/497] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 23 16:54:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:03.932554 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e990ef7-53d6-4979-a41c-dd1411511547 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:03.932554 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e990ef7-53d6-4979-a41c-dd1411511547 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:03.932554 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e990ef7-53d6-4979-a41c-dd1411511547 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:03.932554 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 252/498] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:03 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:03.932554 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e990ef7-53d6-4979-a41c-dd1411511547 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:03.964667 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:03.990397 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b958b186-0d16-414b-b113-8040bdfa934f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:04.027064 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54130b2-4b56-4c9f-abc8-a44b285ce6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:04.027773 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54130b2-4b56-4c9f-abc8-a44b285ce6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:04.028057 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54130b2-4b56-4c9f-abc8-a44b285ce6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:04.028310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:04.028421 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f54130b2-4b56-4c9f-abc8-a44b285ce6f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:04.029045 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54130b2-4b56-4c9f-abc8-a44b285ce6f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:04.029358 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54130b2-4b56-4c9f-abc8-a44b285ce6f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:04.030975 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f54130b2-4b56-4c9f-abc8-a44b285ce6f2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:04.047636 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 247/499] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:04.047636 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54130b2-4b56-4c9f-abc8-a44b285ce6f2 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:04.049779 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e990ef7-53d6-4979-a41c-dd1411511547 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:04.083565 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b2160-e234-4cbf-969a-44165cf5044c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:04.084212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b2160-e234-4cbf-969a-44165cf5044c None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:04.084490 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b2160-e234-4cbf-969a-44165cf5044c None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:04.084762 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41b2160-e234-4cbf-969a-44165cf5044c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:04.145299 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f54130b2-4b56-4c9f-abc8-a44b285ce6f2 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:04.152565 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b2160-e234-4cbf-969a-44165cf5044c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:04.154038 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41b2160-e234-4cbf-969a-44165cf5044c None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:04.180695 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d43d79-915a-4833-baec-ca90ace1588a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:04.181420 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d43d79-915a-4833-baec-ca90ace1588a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:04.181687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d43d79-915a-4833-baec-ca90ace1588a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:04.181963 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d43d79-915a-4833-baec-ca90ace1588a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:04.183921 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:04.193164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d43d79-915a-4833-baec-ca90ace1588a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:04.194637 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d43d79-915a-4833-baec-ca90ace1588a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:04.212130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-a4d43d79-915a-4833-baec-ca90ace1588a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:04.212130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:04.212130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:04.212130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:04.212130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:04.212130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:04.212130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:04.212130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:04.212130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:04.212130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:04.212130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:04.212130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:04.212130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:04.212130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:04.212130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:04.213147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:04.213147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:04.213147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:04.213147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:04.213147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:04.213147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:04.213147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:04.213147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:04.213147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:04.213147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:04.213147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:04.213147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:04.213147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:04.213147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:04.213147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:04.213147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:04.213147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:04.214003 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:04.214003 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:04.214003 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:04.214003 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:04.214003 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 248/500] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:04.214581 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec33df-c384-410d-abe1-f33b8f627114 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:04.214884 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec33df-c384-410d-abe1-f33b8f627114 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:04.215156 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ec33df-c384-410d-abe1-f33b8f627114 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:04.236714 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-e6ec33df-c384-410d-abe1-f33b8f627114 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:04.237936 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e6ec33df-c384-410d-abe1-f33b8f627114 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:04.238078 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:04.258848 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 253/501] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:54:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:54:04.259786 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b2160-e234-4cbf-969a-44165cf5044c None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:04.304816 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 249/502] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:54:04.306206 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8501bf3-a1a1-48fe-848c-4b29d49a2477 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:04.306490 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8501bf3-a1a1-48fe-848c-4b29d49a2477 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:04.306760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8501bf3-a1a1-48fe-848c-4b29d49a2477 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:04.308060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 250/503] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:04.308740 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8501bf3-a1a1-48fe-848c-4b29d49a2477 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:04.375031 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41b2160-e234-4cbf-969a-44165cf5044c None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:04.407751 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46954142-ca27-4bbe-a351-a3831968a72b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:04.408541 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46954142-ca27-4bbe-a351-a3831968a72b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:04.408830 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46954142-ca27-4bbe-a351-a3831968a72b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:04.409095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46954142-ca27-4bbe-a351-a3831968a72b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:04.413533 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8501bf3-a1a1-48fe-848c-4b29d49a2477 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:04.416382 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46954142-ca27-4bbe-a351-a3831968a72b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:04.417849 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46954142-ca27-4bbe-a351-a3831968a72b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:04.430044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:04.432745 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-46954142-ca27-4bbe-a351-a3831968a72b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 23 16:54:04.432745 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:04.432745 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:04.432745 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:04.432745 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:04.432745 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:04.432745 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:04.432745 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:04.432745 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:04.432745 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:04.432745 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:04.432745 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:04.432745 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:04.432745 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:04.432745 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:04.432745 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:04.433827 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:04.433827 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:04.433827 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:04.433827 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:04.433827 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:04.433827 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:04.433827 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:04.433827 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:04.433827 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:04.433827 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:04.433827 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:04.433827 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:04.433827 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:04.433827 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:04.433827 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:04.433827 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:04.433827 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:04.433827 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:04.435001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:04.435001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:04.435001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:04.435001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:04.435001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:04.435001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:04.435001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:04.435001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:04.435001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 23 16:54:04.435001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:04.435001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 254/504] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:54:04 2022] GET /identity/v3/users/glance => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:04.435001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46954142-ca27-4bbe-a351-a3831968a72b None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:04.450878 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655e790a-fe09-417d-973d-0f48b63fbc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:04.451578 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e790a-fe09-417d-973d-0f48b63fbc58 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:04.451861 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e790a-fe09-417d-973d-0f48b63fbc58 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:04.452135 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655e790a-fe09-417d-973d-0f48b63fbc58 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:04.452867 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e790a-fe09-417d-973d-0f48b63fbc58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:04.453171 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e790a-fe09-417d-973d-0f48b63fbc58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:04.454571 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655e790a-fe09-417d-973d-0f48b63fbc58 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:04.466257 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 251/505] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:04.467577 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e62626-99f2-4bfb-85c3-9231a166b33a None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:04.467872 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e62626-99f2-4bfb-85c3-9231a166b33a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:04.468150 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e62626-99f2-4bfb-85c3-9231a166b33a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:04.484542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-d5e62626-99f2-4bfb-85c3-9231a166b33a None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:04.485723 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d5e62626-99f2-4bfb-85c3-9231a166b33a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:04.488613 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:04.545762 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46954142-ca27-4bbe-a351-a3831968a72b None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:04.553974 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 252/506] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:54:04.554823 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e62626-99f2-4bfb-85c3-9231a166b33a None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:04.582982 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43596e-5790-4ec4-8679-90a3e5666b2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:04.583786 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43596e-5790-4ec4-8679-90a3e5666b2f None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:04.584074 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43596e-5790-4ec4-8679-90a3e5666b2f None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:04.584343 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa43596e-5790-4ec4-8679-90a3e5666b2f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:04.594307 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43596e-5790-4ec4-8679-90a3e5666b2f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:04.595772 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa43596e-5790-4ec4-8679-90a3e5666b2f None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:04.613082 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-aa43596e-5790-4ec4-8679-90a3e5666b2f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:04.613082 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:04.613082 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:04.613082 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:04.613082 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:04.613082 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:04.613082 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:04.613082 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:04.613082 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:04.613082 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:04.613082 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:04.613082 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:04.613082 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:04.613082 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:04.613082 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:04.614150 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:04.614150 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:04.614150 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:04.614150 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:04.614150 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:04.614150 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:04.614150 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:04.614150 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:04.614150 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:04.614150 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:04.614150 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:04.614150 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:04.614150 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:04.614150 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:04.614150 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:04.614150 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:04.614150 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:04.615210 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:04.615210 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:04.615210 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:04.615210 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:04.615210 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 255/507] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:04.615210 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43596e-5790-4ec4-8679-90a3e5666b2f None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:04.660210 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e62626-99f2-4bfb-85c3-9231a166b33a None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:04.667369 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:04.696085 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07627e8-1108-431e-b2fe-c4e5d3b607c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:04.696809 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07627e8-1108-431e-b2fe-c4e5d3b607c5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:04.697075 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07627e8-1108-431e-b2fe-c4e5d3b607c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:04.697339 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07627e8-1108-431e-b2fe-c4e5d3b607c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:04.708283 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07627e8-1108-431e-b2fe-c4e5d3b607c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:04.709561 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:04.715888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-d07627e8-1108-431e-b2fe-c4e5d3b607c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:04.715888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:04.715888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:04.715888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:04.715888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:04.715888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:04.715888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:04.715888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:04.715888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:04.715888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:04.715888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:04.715888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:04.715888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:04.715888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:04.715888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:04.715888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:04.716848 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:04.716848 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:04.716848 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:04.716848 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:04.716848 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:04.716848 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:04.716848 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:04.716848 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:04.716848 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:04.716848 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:04.716848 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:04.716848 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:04.716848 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:04.716848 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:04.716848 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:04.716848 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:04.717653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:04.717653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:04.717653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:04.717653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:04.717653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:04.717653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:04.717653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:04.717653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:04.717653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:04.717653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:04.717653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:04.717653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:04.717653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 253/508] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 23 16:54:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:04.718303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa43596e-5790-4ec4-8679-90a3e5666b2f None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:04.718370 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07627e8-1108-431e-b2fe-c4e5d3b607c5 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:04.754396 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ad452-85e8-4897-82eb-0cc18ec0b8af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:04.755097 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ad452-85e8-4897-82eb-0cc18ec0b8af None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:04.755378 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ad452-85e8-4897-82eb-0cc18ec0b8af None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:04.755651 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086ad452-85e8-4897-82eb-0cc18ec0b8af None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:04.756488 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086ad452-85e8-4897-82eb-0cc18ec0b8af None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:04.756801 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086ad452-85e8-4897-82eb-0cc18ec0b8af None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:04.758240 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086ad452-85e8-4897-82eb-0cc18ec0b8af None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:04.794895 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 256/509] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 23 16:54:04 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:04.795722 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ad452-85e8-4897-82eb-0cc18ec0b8af None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:04.820349 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07627e8-1108-431e-b2fe-c4e5d3b607c5 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:04.854613 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5efcf-5d12-48ad-bc30-8a6fbae885db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:04.855513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5efcf-5d12-48ad-bc30-8a6fbae885db None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:04.855801 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5efcf-5d12-48ad-bc30-8a6fbae885db None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:04.856075 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b5efcf-5d12-48ad-bc30-8a6fbae885db None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:04.863838 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b5efcf-5d12-48ad-bc30-8a6fbae885db None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:04.865290 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5b5efcf-5d12-48ad-bc30-8a6fbae885db None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:04.880701 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-c5b5efcf-5d12-48ad-bc30-8a6fbae885db None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 23 16:54:04.880701 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:04.880701 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:04.880701 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:04.880701 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:04.880701 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:04.880701 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:04.880701 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:04.880701 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:04.880701 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:04.880701 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:04.880701 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:04.880701 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:04.880701 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:04.880701 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:04.880701 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:04.881611 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:04.881611 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:04.881611 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:04.881611 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:04.881611 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:04.881611 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:04.881611 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:04.881611 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:04.881611 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:04.881611 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:04.881611 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:04.881611 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:04.881611 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:04.881611 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:04.881611 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:04.881611 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:04.881611 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:04.881611 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:04.882505 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:04.882505 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:04.882505 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:04.882505 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:04.882505 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:04.882505 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:04.882505 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:04.882505 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:04.882505 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 23 16:54:04.882505 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:04.882505 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 254/510] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 23 16:54:04 2022] GET /identity/v3/users/neutron => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:04.883088 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5efcf-5d12-48ad-bc30-8a6fbae885db None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:04.905679 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086ad452-85e8-4897-82eb-0cc18ec0b8af None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:04.911324 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:04.942283 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20844463-8899-432b-8a25-188c72213046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:04.942754 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20844463-8899-432b-8a25-188c72213046 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:04.943070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20844463-8899-432b-8a25-188c72213046 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:04.943353 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20844463-8899-432b-8a25-188c72213046 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:04.953991 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20844463-8899-432b-8a25-188c72213046 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:04.955448 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20844463-8899-432b-8a25-188c72213046 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:04.973844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-20844463-8899-432b-8a25-188c72213046 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:04.973844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:04.973844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:04.973844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:04.973844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:04.973844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:04.973844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:04.973844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:04.973844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:04.973844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:04.973844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:04.973844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:04.973844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:04.973844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:04.973844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:04.974850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:04.974850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:04.974850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:04.974850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:04.974850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:04.974850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:04.974850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:04.974850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:04.974850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:04.974850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:04.974850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:04.974850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:04.974850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:04.974850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:04.974850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:04.974850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:04.974850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:04.975773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:04.975773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:04.975773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:04.975773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:04.975773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 257/511] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:04.976039 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20844463-8899-432b-8a25-188c72213046 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:04.979941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:04.986132 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b5efcf-5d12-48ad-bc30-8a6fbae885db None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:05.020233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07064ef-a0d3-453b-bae1-f4a40e532b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:05.020887 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07064ef-a0d3-453b-bae1-f4a40e532b96 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:05.021171 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07064ef-a0d3-453b-bae1-f4a40e532b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:05.021491 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07064ef-a0d3-453b-bae1-f4a40e532b96 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:05.022237 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07064ef-a0d3-453b-bae1-f4a40e532b96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:05.022540 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07064ef-a0d3-453b-bae1-f4a40e532b96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:05.023970 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07064ef-a0d3-453b-bae1-f4a40e532b96 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:05.036256 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 255/512] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:05.036880 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07064ef-a0d3-453b-bae1-f4a40e532b96 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:05.082874 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20844463-8899-432b-8a25-188c72213046 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:05.117857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0b934-07a5-4db9-aeab-0bbca6e48b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:05.118507 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0b934-07a5-4db9-aeab-0bbca6e48b9c None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:05.118751 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:05.118877 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0b934-07a5-4db9-aeab-0bbca6e48b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:05.119082 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0b934-07a5-4db9-aeab-0bbca6e48b9c None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:05.119832 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c0b934-07a5-4db9-aeab-0bbca6e48b9c None admin] RBAC: Adding query filter params (name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:05.120136 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c0b934-07a5-4db9-aeab-0bbca6e48b9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:05.122089 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c0b934-07a5-4db9-aeab-0bbca6e48b9c None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:05.129991 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 258/513] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 23 16:54:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:05.130774 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0b934-07a5-4db9-aeab-0bbca6e48b9c None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:05.137714 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07064ef-a0d3-453b-bae1-f4a40e532b96 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:05.172259 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6718d-4c0d-4f74-aa31-8c3d4aa3213f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:05.172912 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6718d-4c0d-4f74-aa31-8c3d4aa3213f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:05.173194 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6718d-4c0d-4f74-aa31-8c3d4aa3213f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:05.173474 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb6718d-4c0d-4f74-aa31-8c3d4aa3213f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:05.197011 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:05.232134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c0b934-07a5-4db9-aeab-0bbca6e48b9c None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:05.242238 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6718d-4c0d-4f74-aa31-8c3d4aa3213f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:05.243745 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb6718d-4c0d-4f74-aa31-8c3d4aa3213f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:05.266013 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac1aee-51bc-4436-abf5-df650a80a7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:05.266757 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac1aee-51bc-4436-abf5-df650a80a7be None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:05.267087 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac1aee-51bc-4436-abf5-df650a80a7be None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:05.267364 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ac1aee-51bc-4436-abf5-df650a80a7be None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:05.268204 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ac1aee-51bc-4436-abf5-df650a80a7be None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:05.268515 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ac1aee-51bc-4436-abf5-df650a80a7be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:05.270002 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ac1aee-51bc-4436-abf5-df650a80a7be None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:05.286772 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 259/514] 10.222.0.13 () {56 vars in 1139 bytes} [Mon May 23 16:54:05 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:05.287613 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac1aee-51bc-4436-abf5-df650a80a7be None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:05.328901 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:05.344878 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 256/515] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:54:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:54:05.347792 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6718d-4c0d-4f74-aa31-8c3d4aa3213f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:05.389812 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ac1aee-51bc-4436-abf5-df650a80a7be None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:05.413533 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:05.426526 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d019d5-e305-4c13-91ce-5c7c8bb5650f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:05.427224 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d019d5-e305-4c13-91ce-5c7c8bb5650f None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:05.427512 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d019d5-e305-4c13-91ce-5c7c8bb5650f None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:05.427780 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d019d5-e305-4c13-91ce-5c7c8bb5650f None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:05.428498 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d019d5-e305-4c13-91ce-5c7c8bb5650f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:05.428799 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d019d5-e305-4c13-91ce-5c7c8bb5650f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:05.430181 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d019d5-e305-4c13-91ce-5c7c8bb5650f None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:05.441393 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 260/516] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:05.446573 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d019d5-e305-4c13-91ce-5c7c8bb5650f None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:05.446683 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb6718d-4c0d-4f74-aa31-8c3d4aa3213f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:05.483890 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7623d-c1bd-4733-bc3b-5034e8152969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:05.484632 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7623d-c1bd-4733-bc3b-5034e8152969 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:05.484914 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7623d-c1bd-4733-bc3b-5034e8152969 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:05.485190 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7623d-c1bd-4733-bc3b-5034e8152969 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:05.492441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7623d-c1bd-4733-bc3b-5034e8152969 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:05.493919 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7623d-c1bd-4733-bc3b-5034e8152969 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:05.508932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-34a7623d-c1bd-4733-bc3b-5034e8152969 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:05.508932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:05.508932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:05.508932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:05.508932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:05.508932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:05.508932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:05.508932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:05.508932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:05.508932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:05.508932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:05.508932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:05.508932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:05.508932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:05.508932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:05.509889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:05.509889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:05.509889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:05.509889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:05.509889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:05.509889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:05.509889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:05.509889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:05.509889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:05.509889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:05.509889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:05.509889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:05.509889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:05.509889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:05.509889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:05.509889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:05.509889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:05.510743 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:05.510743 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:05.510743 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:05.510743 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:05.510743 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:05.510743 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:05.510743 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:05.510743 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:05.510743 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 257/517] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:05.511259 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7623d-c1bd-4733-bc3b-5034e8152969 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:05.563691 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:05.564007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d019d5-e305-4c13-91ce-5c7c8bb5650f None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:05.599081 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382682d-8054-40c7-9b80-8f7f8e20184d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:05.599871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382682d-8054-40c7-9b80-8f7f8e20184d None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:05.600151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382682d-8054-40c7-9b80-8f7f8e20184d None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:05.600424 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382682d-8054-40c7-9b80-8f7f8e20184d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:05.611670 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382682d-8054-40c7-9b80-8f7f8e20184d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:05.613147 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7382682d-8054-40c7-9b80-8f7f8e20184d None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:05.617137 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7623d-c1bd-4733-bc3b-5034e8152969 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:05.630212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-7382682d-8054-40c7-9b80-8f7f8e20184d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:05.630212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:05.630212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:05.630212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:05.630212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:05.630212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:05.630212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:05.630212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:05.630212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:05.630212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:05.630212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:05.630212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:05.630212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:05.630212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:05.630212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:05.631174 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:05.631174 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:05.631174 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:05.631174 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:05.631174 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:05.631174 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:05.631174 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:05.631174 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:05.631174 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:05.631174 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:05.631174 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:05.631174 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:05.631174 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:05.631174 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:05.631174 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:05.631174 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:05.631174 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:05.632022 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:05.632022 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:05.632022 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:05.632022 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:05.632022 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 261/518] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:05.632688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382682d-8054-40c7-9b80-8f7f8e20184d None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:05.655517 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96854dc-1bba-45a9-b827-b510e878f83b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:05.656160 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96854dc-1bba-45a9-b827-b510e878f83b None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:05.656442 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96854dc-1bba-45a9-b827-b510e878f83b None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:05.656708 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96854dc-1bba-45a9-b827-b510e878f83b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:05.670609 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:05.723478 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96854dc-1bba-45a9-b827-b510e878f83b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:05.724957 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96854dc-1bba-45a9-b827-b510e878f83b None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:05.737010 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7382682d-8054-40c7-9b80-8f7f8e20184d None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:05.769509 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:05.771244 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2e1cf-d5ae-4378-a889-79c45b26e1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:05.772142 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2e1cf-d5ae-4378-a889-79c45b26e1de None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:05.772420 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2e1cf-d5ae-4378-a889-79c45b26e1de None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:05.772691 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea2e1cf-d5ae-4378-a889-79c45b26e1de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:05.783095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea2e1cf-d5ae-4378-a889-79c45b26e1de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:05.784568 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea2e1cf-d5ae-4378-a889-79c45b26e1de None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:05.802215 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-5ea2e1cf-d5ae-4378-a889-79c45b26e1de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:05.802215 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:05.802215 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:05.802215 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:05.802215 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:05.802215 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:05.802215 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:05.802215 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:05.802215 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:05.802215 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:05.802215 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:05.802215 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:05.802215 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:05.802215 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:05.802215 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:05.803106 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:05.803106 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:05.803106 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:05.803106 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:05.803106 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:05.803106 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:05.803106 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:05.803106 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:05.803106 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:05.803106 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:05.803106 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:05.803106 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:05.803106 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:05.803106 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:05.803106 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:05.803106 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:05.803106 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:05.808814 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:05.808814 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:05.808814 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:05.808814 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:05.808814 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 262/519] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:05.808814 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2e1cf-d5ae-4378-a889-79c45b26e1de None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:05.824764 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 258/520] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:54:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:54:05.826335 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96854dc-1bba-45a9-b827-b510e878f83b None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:05.902199 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:05.916562 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea2e1cf-d5ae-4378-a889-79c45b26e1de None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:05.934427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96854dc-1bba-45a9-b827-b510e878f83b None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:05.950399 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2608de3-b75f-4570-bba4-6989b3d4eb30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:05.951104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2608de3-b75f-4570-bba4-6989b3d4eb30 None admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:05.951387 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2608de3-b75f-4570-bba4-6989b3d4eb30 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:05.951659 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2608de3-b75f-4570-bba4-6989b3d4eb30 None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:05.952352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2608de3-b75f-4570-bba4-6989b3d4eb30 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:05.953783 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2608de3-b75f-4570-bba4-6989b3d4eb30 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:05.968415 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff8c2f-eafd-42ce-b3f2-7bd841da90e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:05.969077 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff8c2f-eafd-42ce-b3f2-7bd841da90e5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:05.969350 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff8c2f-eafd-42ce-b3f2-7bd841da90e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:05.969622 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ff8c2f-eafd-42ce-b3f2-7bd841da90e5 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:05.970535 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ff8c2f-eafd-42ce-b3f2-7bd841da90e5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:05.970840 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ff8c2f-eafd-42ce-b3f2-7bd841da90e5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:05.972386 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ff8c2f-eafd-42ce-b3f2-7bd841da90e5 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:05.974019 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 263/521] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 23 16:54:05 2022] POST /identity/v3/users => generated 247 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:54:05.974844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2608de3-b75f-4570-bba4-6989b3d4eb30 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:05.993783 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 259/522] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:05.996459 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff8c2f-eafd-42ce-b3f2-7bd841da90e5 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:06.006492 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:06.090431 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2608de3-b75f-4570-bba4-6989b3d4eb30 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:06.116082 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ff8c2f-eafd-42ce-b3f2-7bd841da90e5 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:06.128650 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca03942-db01-4803-9134-a6d65b1187b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:06.129370 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca03942-db01-4803-9134-a6d65b1187b6 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:06.129662 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca03942-db01-4803-9134-a6d65b1187b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:06.129938 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca03942-db01-4803-9134-a6d65b1187b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:06.130849 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca03942-db01-4803-9134-a6d65b1187b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:06.131189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca03942-db01-4803-9134-a6d65b1187b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:06.132603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca03942-db01-4803-9134-a6d65b1187b6 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:06.150821 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 264/523] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:06.150821 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca03942-db01-4803-9134-a6d65b1187b6 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:06.153438 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:06.159241 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3cd621-4a7b-4e01-b9ff-7afde58194c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:06.159919 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cd621-4a7b-4e01-b9ff-7afde58194c9 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:06.160198 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cd621-4a7b-4e01-b9ff-7afde58194c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:06.160523 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3cd621-4a7b-4e01-b9ff-7afde58194c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:06.161708 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3cd621-4a7b-4e01-b9ff-7afde58194c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:06.161708 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3cd621-4a7b-4e01-b9ff-7afde58194c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:06.162987 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3cd621-4a7b-4e01-b9ff-7afde58194c9 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:06.177602 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 260/524] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:06.178732 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3cd621-4a7b-4e01-b9ff-7afde58194c9 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:06.262093 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:06.273068 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca03942-db01-4803-9134-a6d65b1187b6 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:06.300418 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3cd621-4a7b-4e01-b9ff-7afde58194c9 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:06.306656 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eadda02-035d-44f1-a3ca-82dfa9394f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:06.307504 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eadda02-035d-44f1-a3ca-82dfa9394f3a None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:06.307787 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eadda02-035d-44f1-a3ca-82dfa9394f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:06.308055 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eadda02-035d-44f1-a3ca-82dfa9394f3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:06.320563 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eadda02-035d-44f1-a3ca-82dfa9394f3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:06.322450 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eadda02-035d-44f1-a3ca-82dfa9394f3a None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:06.337553 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c446e-73a4-40c4-8f5b-e10fb47db197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:06.338193 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c446e-73a4-40c4-8f5b-e10fb47db197 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:06.338488 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c446e-73a4-40c4-8f5b-e10fb47db197 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:06.338757 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035c446e-73a4-40c4-8f5b-e10fb47db197 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:06.339606 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c446e-73a4-40c4-8f5b-e10fb47db197 None admin] RBAC: Adding query filter params (role.id=5a7214dc069f45969e19c44122b378a4, scope.project.id=0595667fb2e04a3ca4716f669d6698b0, user.id=225c7b385f12445fab86b3b5606b1da1) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:06.339909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c446e-73a4-40c4-8f5b-e10fb47db197 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:06.340239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-6eadda02-035d-44f1-a3ca-82dfa9394f3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:06.340239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:06.340239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:06.340239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:06.340239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:06.340239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:06.340239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:06.340239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:06.340239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:06.340239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:06.340239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:06.340239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:06.340239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:06.340239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:06.340239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:06.341101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:06.341101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:06.341101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:06.341101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:06.341101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:06.341101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:06.341101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:06.341101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:06.341101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:06.341101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:06.341101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:06.341101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:06.341101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:06.341101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:06.341101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:06.341101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:06.341101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:06.341941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:06.341941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:06.341941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:06.341941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:06.342176 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-035c446e-73a4-40c4-8f5b-e10fb47db197 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:06.342244 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 265/525] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:06.342720 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eadda02-035d-44f1-a3ca-82dfa9394f3a None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:06.354952 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 261/526] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:54:06 2022] GET /identity/v3/role_assignments?user.id=225c7b385f12445fab86b3b5606b1da1&scope.project.id=0595667fb2e04a3ca4716f669d6698b0&role.id=5a7214dc069f45969e19c44122b378a4 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:06.354952 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c446e-73a4-40c4-8f5b-e10fb47db197 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:06.407185 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:06.458366 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eadda02-035d-44f1-a3ca-82dfa9394f3a None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:06.475251 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-035c446e-73a4-40c4-8f5b-e10fb47db197 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:06.502212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d002143-6bc4-404b-a71d-e8476130001a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:06.503118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d002143-6bc4-404b-a71d-e8476130001a None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:06.503404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d002143-6bc4-404b-a71d-e8476130001a None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:06.503688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d002143-6bc4-404b-a71d-e8476130001a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:06.512780 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d002143-6bc4-404b-a71d-e8476130001a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:06.514231 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d002143-6bc4-404b-a71d-e8476130001a None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:06.514992 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:06.524178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859ce65-87e5-49de-a6ba-e12fdd28f735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:06.525338 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859ce65-87e5-49de-a6ba-e12fdd28f735 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:06.525807 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859ce65-87e5-49de-a6ba-e12fdd28f735 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:06.526371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d859ce65-87e5-49de-a6ba-e12fdd28f735 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:06.531161 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-5d002143-6bc4-404b-a71d-e8476130001a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 16:54:06.531161 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:06.531161 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:06.531161 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:06.531161 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:06.531161 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:06.531161 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:06.531161 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:06.531161 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:06.531161 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:06.531161 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:06.531161 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:06.531161 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:06.531161 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:06.531161 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:06.531161 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:06.532101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:06.532101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:06.532101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:06.532101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:06.532101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:06.532101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:06.532101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:06.532101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:06.532101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:06.532101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:06.532101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:06.532101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:06.532101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:06.532101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:06.532101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:06.532101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:06.532101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:06.532101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:06.532981 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:06.532981 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:06.532981 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:06.532981 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:06.532981 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:06.532981 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:06.532981 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:06.532981 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:06.532981 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 16:54:06.532981 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:06.537657 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 266/527] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:54:06 2022] GET /identity/v3/users/cinder => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:06.538653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d859ce65-87e5-49de-a6ba-e12fdd28f735 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:06.538785 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d002143-6bc4-404b-a71d-e8476130001a None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:06.540668 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d859ce65-87e5-49de-a6ba-e12fdd28f735 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:06.563410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-d859ce65-87e5-49de-a6ba-e12fdd28f735 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:06.563410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:06.563410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:06.563410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:06.563410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:06.563410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:06.563410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:06.563410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:06.563410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:06.563410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:06.563410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:06.563410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:06.563410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:06.563410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:06.563410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:06.564339 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:06.564339 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:06.564339 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:06.564339 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:06.564339 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:06.564339 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:06.564339 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:06.564339 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:06.564339 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:06.564339 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:06.564339 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:06.564339 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:06.564339 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:06.564339 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:06.564339 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:06.564339 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:06.564339 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:06.565167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:06.565167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:06.565167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:06.565167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:06.565167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:06.565167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:06.565167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:06.565167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:06.570303 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 262/528] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:06.571362 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859ce65-87e5-49de-a6ba-e12fdd28f735 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:06.654871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d002143-6bc4-404b-a71d-e8476130001a None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:06.671949 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:06.693732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991359fb-26f3-412a-91bc-029a87498503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:06.693732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991359fb-26f3-412a-91bc-029a87498503 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:06.693732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991359fb-26f3-412a-91bc-029a87498503 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:06.693732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991359fb-26f3-412a-91bc-029a87498503 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:06.694127 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991359fb-26f3-412a-91bc-029a87498503 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:06.694127 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991359fb-26f3-412a-91bc-029a87498503 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:06.696071 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991359fb-26f3-412a-91bc-029a87498503 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:06.708076 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d859ce65-87e5-49de-a6ba-e12fdd28f735 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:06.708804 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 267/529] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:06.709755 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991359fb-26f3-412a-91bc-029a87498503 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:06.754834 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78457cbc-521e-44be-a2ba-a4808327e90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:06.755571 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78457cbc-521e-44be-a2ba-a4808327e90d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:06.755856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78457cbc-521e-44be-a2ba-a4808327e90d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:06.756130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78457cbc-521e-44be-a2ba-a4808327e90d None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:06.757244 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78457cbc-521e-44be-a2ba-a4808327e90d None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:06.757559 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78457cbc-521e-44be-a2ba-a4808327e90d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:06.760259 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78457cbc-521e-44be-a2ba-a4808327e90d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:06.783545 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 263/530] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 23 16:54:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:06.784975 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78457cbc-521e-44be-a2ba-a4808327e90d None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:06.831147 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991359fb-26f3-412a-91bc-029a87498503 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:06.848789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:06.869192 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3955e-2dcc-488e-88b1-b71ef9cb97f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:06.869872 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3955e-2dcc-488e-88b1-b71ef9cb97f2 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:06.870161 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3955e-2dcc-488e-88b1-b71ef9cb97f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:06.870439 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a3955e-2dcc-488e-88b1-b71ef9cb97f2 None admin] PATH_INFO: `/v3/projects` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:06.871550 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3955e-2dcc-488e-88b1-b71ef9cb97f2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:06.871865 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3955e-2dcc-488e-88b1-b71ef9cb97f2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:06.873475 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a3955e-2dcc-488e-88b1-b71ef9cb97f2 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:06.902937 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 268/531] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:06.902937 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3955e-2dcc-488e-88b1-b71ef9cb97f2 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:06.911588 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78457cbc-521e-44be-a2ba-a4808327e90d None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:06.942351 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:06.946275 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8275e-93b7-456f-bc26-a8df2d16b98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:06.947202 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8275e-93b7-456f-bc26-a8df2d16b98f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:06.947548 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8275e-93b7-456f-bc26-a8df2d16b98f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:06.947889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e8275e-93b7-456f-bc26-a8df2d16b98f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:06.955593 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e8275e-93b7-456f-bc26-a8df2d16b98f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:06.957457 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e8275e-93b7-456f-bc26-a8df2d16b98f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:06.972529 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-e4e8275e-93b7-456f-bc26-a8df2d16b98f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:06.972529 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:06.972529 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:06.972529 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:06.972529 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:06.972529 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:06.972529 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:06.972529 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:06.972529 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:06.972529 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:06.972529 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:06.972529 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:06.972529 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:06.972529 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:06.972529 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:06.973422 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:06.973422 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:06.973422 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:06.973422 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:06.973422 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:06.973422 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:06.973422 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:06.973422 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:06.973422 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:06.973422 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:06.973422 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:06.973422 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:06.973422 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:06.973422 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:06.973422 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:06.973422 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:06.973422 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:06.974249 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:06.974249 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:06.974249 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:06.974249 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:06.974249 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:06.974249 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:06.974249 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:06.974249 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:06.974249 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 264/532] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:06.975547 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8275e-93b7-456f-bc26-a8df2d16b98f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:07.018713 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a3955e-2dcc-488e-88b1-b71ef9cb97f2 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:07.061627 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e17cf-8455-467c-a413-8f3fff9b47dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:07.062432 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e17cf-8455-467c-a413-8f3fff9b47dd None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:07.062721 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e17cf-8455-467c-a413-8f3fff9b47dd None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:07.063381 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e17cf-8455-467c-a413-8f3fff9b47dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:07.091595 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e8275e-93b7-456f-bc26-a8df2d16b98f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:07.101837 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:07.127844 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f3f01-3564-4891-a524-b9cb52217aac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:07.128513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f3f01-3564-4891-a524-b9cb52217aac None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:07.128798 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f3f01-3564-4891-a524-b9cb52217aac None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:07.129082 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1f3f01-3564-4891-a524-b9cb52217aac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:07.130424 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f3f01-3564-4891-a524-b9cb52217aac None admin] RBAC: Adding query filter params (role.id=5a7214dc069f45969e19c44122b378a4, scope.project.id=0595667fb2e04a3ca4716f669d6698b0, user.id=d93036329cf64a8e91edaee5b1d47b9c) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:07.130424 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f3f01-3564-4891-a524-b9cb52217aac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:07.131940 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1f3f01-3564-4891-a524-b9cb52217aac None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:07.147400 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 265/533] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:54:06 2022] GET /identity/v3/role_assignments?user.id=d93036329cf64a8e91edaee5b1d47b9c&scope.project.id=0595667fb2e04a3ca4716f669d6698b0&role.id=5a7214dc069f45969e19c44122b378a4 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:07.148541 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f3f01-3564-4891-a524-b9cb52217aac None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:07.150812 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e17cf-8455-467c-a413-8f3fff9b47dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:07.152910 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b43e17cf-8455-467c-a413-8f3fff9b47dd None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:07.223892 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:07.271919 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 269/534] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:54:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:54:07.281321 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e17cf-8455-467c-a413-8f3fff9b47dd None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:07.388614 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1f3f01-3564-4891-a524-b9cb52217aac None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:07.425158 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446fb31-80a2-4681-9696-188b0faaedad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:07.425824 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446fb31-80a2-4681-9696-188b0faaedad None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:07.426157 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446fb31-80a2-4681-9696-188b0faaedad None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:07.426432 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e446fb31-80a2-4681-9696-188b0faaedad None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:07.427264 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446fb31-80a2-4681-9696-188b0faaedad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:07.427571 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446fb31-80a2-4681-9696-188b0faaedad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:07.429008 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e446fb31-80a2-4681-9696-188b0faaedad None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:07.429819 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b43e17cf-8455-467c-a413-8f3fff9b47dd None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:07.441553 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 266/535] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:07.447452 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446fb31-80a2-4681-9696-188b0faaedad None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:07.472098 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35c78e-8ed6-4d53-a288-82a472b99b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:07.473417 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35c78e-8ed6-4d53-a288-82a472b99b0d None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:07.473712 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35c78e-8ed6-4d53-a288-82a472b99b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:07.473992 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c35c78e-8ed6-4d53-a288-82a472b99b0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:07.480235 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:07.482631 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:07.492546 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35c78e-8ed6-4d53-a288-82a472b99b0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:07.494225 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c35c78e-8ed6-4d53-a288-82a472b99b0d None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:07.523968 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-9c35c78e-8ed6-4d53-a288-82a472b99b0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:07.523968 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:07.523968 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:07.523968 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:07.523968 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:07.523968 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:07.523968 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:07.523968 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:07.523968 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:07.523968 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:07.523968 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:07.523968 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:07.523968 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:07.523968 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:07.523968 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:07.524871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:07.524871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:07.524871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:07.524871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:07.524871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:07.524871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:07.524871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:07.524871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:07.524871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:07.524871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:07.524871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:07.524871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:07.524871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:07.524871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:07.524871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:07.524871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:07.524871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:07.525808 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:07.525808 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:07.525808 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:07.525808 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:07.525808 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 270/536] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:07.531771 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35c78e-8ed6-4d53-a288-82a472b99b0d None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:07.621123 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e446fb31-80a2-4681-9696-188b0faaedad None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:07.669011 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aeefd8-9c69-4be1-a999-fbb07696ae00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:07.669586 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aeefd8-9c69-4be1-a999-fbb07696ae00 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:07.669868 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aeefd8-9c69-4be1-a999-fbb07696ae00 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:07.670137 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03aeefd8-9c69-4be1-a999-fbb07696ae00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:07.670987 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aeefd8-9c69-4be1-a999-fbb07696ae00 None admin] RBAC: Adding query filter params (role.id=5a7214dc069f45969e19c44122b378a4, scope.project.id=0595667fb2e04a3ca4716f669d6698b0, user.id=fd070e7581f04f4abcbb8b6bc231a8c1) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:07.674142 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aeefd8-9c69-4be1-a999-fbb07696ae00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:07.675687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03aeefd8-9c69-4be1-a999-fbb07696ae00 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:07.685731 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 267/537] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:54:07 2022] GET /identity/v3/role_assignments?user.id=fd070e7581f04f4abcbb8b6bc231a8c1&scope.project.id=0595667fb2e04a3ca4716f669d6698b0&role.id=5a7214dc069f45969e19c44122b378a4 => generated 259 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:07.693476 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c35c78e-8ed6-4d53-a288-82a472b99b0d None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:07.730655 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ee3fb-e3dc-490d-9ca4-c47702718dc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:07.731375 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ee3fb-e3dc-490d-9ca4-c47702718dc8 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:07.731656 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ee3fb-e3dc-490d-9ca4-c47702718dc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:07.731928 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740ee3fb-e3dc-490d-9ca4-c47702718dc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:07.732657 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ee3fb-e3dc-490d-9ca4-c47702718dc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:07.732963 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ee3fb-e3dc-490d-9ca4-c47702718dc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:07.734366 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740ee3fb-e3dc-490d-9ca4-c47702718dc8 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:07.748368 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 271/538] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:07.753662 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aeefd8-9c69-4be1-a999-fbb07696ae00 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:07.920406 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:07.927314 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03aeefd8-9c69-4be1-a999-fbb07696ae00 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:07.980545 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be472b-6fe0-44aa-9271-b03568af81df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:07.981712 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be472b-6fe0-44aa-9271-b03568af81df None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:07.982192 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be472b-6fe0-44aa-9271-b03568af81df None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:07.982655 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9be472b-6fe0-44aa-9271-b03568af81df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:07.996830 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be472b-6fe0-44aa-9271-b03568af81df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:07.998674 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9be472b-6fe0-44aa-9271-b03568af81df None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:08.016988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-e9be472b-6fe0-44aa-9271-b03568af81df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:08.016988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:08.016988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:08.016988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:08.016988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:08.016988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:08.016988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:08.016988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:08.016988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:08.016988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:08.016988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:08.016988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:08.016988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:08.016988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:08.016988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:08.017902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:08.017902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:08.017902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:08.017902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:08.017902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:08.017902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:08.017902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:08.017902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:08.017902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:08.017902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:08.017902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:08.017902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:08.017902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:08.017902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:08.017902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:08.017902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:08.017902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:08.018755 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:08.018755 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:08.018755 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:08.018755 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:08.018755 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:08.018755 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:08.018755 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:08.018755 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:08.021725 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 268/539] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:08.027308 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ee3fb-e3dc-490d-9ca4-c47702718dc8 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:08.101365 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:08.191403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740ee3fb-e3dc-490d-9ca4-c47702718dc8 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:08.253932 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847fd940-8b4b-4caa-9d02-c3a19e14b6c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:08.255499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fd940-8b4b-4caa-9d02-c3a19e14b6c3 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:08.255985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fd940-8b4b-4caa-9d02-c3a19e14b6c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:08.256456 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847fd940-8b4b-4caa-9d02-c3a19e14b6c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:08.257440 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fd940-8b4b-4caa-9d02-c3a19e14b6c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:08.257986 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fd940-8b4b-4caa-9d02-c3a19e14b6c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:08.259973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847fd940-8b4b-4caa-9d02-c3a19e14b6c3 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:08.275111 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 272/540] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:08.280946 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be472b-6fe0-44aa-9271-b03568af81df None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:08.409700 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06273f21-53d9-427d-bf96-eb5b3df268e1 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:08.410293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06273f21-53d9-427d-bf96-eb5b3df268e1 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:08.410834 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06273f21-53d9-427d-bf96-eb5b3df268e1 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:08.412712 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 273/541] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:08.419099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277fbe2-94a5-470a-86fd-32de7c0124b1 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:08.419660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277fbe2-94a5-470a-86fd-32de7c0124b1 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:08.420164 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6277fbe2-94a5-470a-86fd-32de7c0124b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:08.433965 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9be472b-6fe0-44aa-9271-b03568af81df None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:08.452519 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-6277fbe2-94a5-470a-86fd-32de7c0124b1 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:08.453774 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-6277fbe2-94a5-470a-86fd-32de7c0124b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:08.482942 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2758f3-cfed-48d6-9207-82550c1ef310 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:08.482942 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2758f3-cfed-48d6-9207-82550c1ef310 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:08.482942 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2758f3-cfed-48d6-9207-82550c1ef310 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:08.482942 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2758f3-cfed-48d6-9207-82550c1ef310 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:08.483379 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2758f3-cfed-48d6-9207-82550c1ef310 None admin] RBAC: Adding query filter params (role.id=5a7214dc069f45969e19c44122b378a4, scope.project.id=0595667fb2e04a3ca4716f669d6698b0, user.id=22e45ab9d34c4fcbaf93be339257eca8) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:08.483379 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2758f3-cfed-48d6-9207-82550c1ef310 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:08.484871 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2758f3-cfed-48d6-9207-82550c1ef310 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:08.502145 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 269/542] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:54:08 2022] GET /identity/v3/role_assignments?user.id=22e45ab9d34c4fcbaf93be339257eca8&scope.project.id=0595667fb2e04a3ca4716f669d6698b0&role.id=5a7214dc069f45969e19c44122b378a4 => generated 259 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:08.563234 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 274/543] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:54:08.605239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2758f3-cfed-48d6-9207-82550c1ef310 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:08.637648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:08.761730 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2758f3-cfed-48d6-9207-82550c1ef310 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:08.804500 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c2990-7358-44f9-ae19-4499d2ce2ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:08.805308 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c2990-7358-44f9-ae19-4499d2ce2ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:08.805588 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c2990-7358-44f9-ae19-4499d2ce2ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:08.805859 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349c2990-7358-44f9-ae19-4499d2ce2ac2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:08.821300 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349c2990-7358-44f9-ae19-4499d2ce2ac2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:08.829097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-349c2990-7358-44f9-ae19-4499d2ce2ac2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:08.829097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:08.829097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:08.829097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:08.829097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:08.829097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:08.829097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:08.829097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:08.829097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:08.829097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:08.829097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:08.829097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:08.829097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:08.829097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:08.829097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:08.829097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:08.829986 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:08.829986 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:08.829986 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:08.829986 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:08.829986 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:08.829986 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:08.829986 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:08.829986 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:08.829986 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:08.829986 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:08.829986 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:08.829986 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:08.829986 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:08.829986 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:08.829986 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:08.829986 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:08.830782 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:08.830782 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:08.830782 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:08.830782 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:08.830782 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:08.830782 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:08.830782 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:08.830782 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:08.830782 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:08.830782 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:08.830782 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:08.830782 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:08.830782 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 270/544] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 23 16:54:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:08.837456 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277fbe2-94a5-470a-86fd-32de7c0124b1 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:08.897765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:08.968947 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6277fbe2-94a5-470a-86fd-32de7c0124b1 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:09.009347 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318f5e9b-9c81-4338-b0a2-1f80f43b76f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:09.010092 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f5e9b-9c81-4338-b0a2-1f80f43b76f0 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:09.010373 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f5e9b-9c81-4338-b0a2-1f80f43b76f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:09.010645 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318f5e9b-9c81-4338-b0a2-1f80f43b76f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:09.011479 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318f5e9b-9c81-4338-b0a2-1f80f43b76f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:09.011842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318f5e9b-9c81-4338-b0a2-1f80f43b76f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:09.013188 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318f5e9b-9c81-4338-b0a2-1f80f43b76f0 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:09.026979 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 275/545] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 23 16:54:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:09.032518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c2990-7358-44f9-ae19-4499d2ce2ac2 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:09.059046 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6029efad-a4cd-4863-a942-b9f938a9075f None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:09.059340 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6029efad-a4cd-4863-a942-b9f938a9075f None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:09.059595 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6029efad-a4cd-4863-a942-b9f938a9075f None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:09.062670 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 276/546] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:54:09.068286 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011485bb-dd53-4619-8612-27c1fb8ec7f2 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:09.068892 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011485bb-dd53-4619-8612-27c1fb8ec7f2 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:09.069604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011485bb-dd53-4619-8612-27c1fb8ec7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:09.108001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-011485bb-dd53-4619-8612-27c1fb8ec7f2 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:09.110993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-011485bb-dd53-4619-8612-27c1fb8ec7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:09.179482 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:09.208981 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349c2990-7358-44f9-ae19-4499d2ce2ac2 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:09.246954 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfad815-f617-4ea2-9653-06841c8cfc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:09.248181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfad815-f617-4ea2-9653-06841c8cfc28 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:09.248703 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfad815-f617-4ea2-9653-06841c8cfc28 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:09.249183 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfad815-f617-4ea2-9653-06841c8cfc28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:09.267022 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 277/547] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 16:54:09.270766 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfad815-f617-4ea2-9653-06841c8cfc28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:09.272529 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfad815-f617-4ea2-9653-06841c8cfc28 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:09.305574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-8dfad815-f617-4ea2-9653-06841c8cfc28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:09.305574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:09.305574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:09.305574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:09.305574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:09.305574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:09.305574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:09.305574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:09.305574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:09.305574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:09.305574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:09.305574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:09.305574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:09.305574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:09.305574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:09.306503 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:09.306503 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:09.306503 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:09.306503 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:09.306503 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:09.306503 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:09.306503 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:09.306503 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:09.306503 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:09.306503 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:09.306503 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:09.306503 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:09.306503 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:09.306503 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:09.306503 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:09.306503 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:09.306503 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:09.307457 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:09.307457 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:09.307457 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:09.307457 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:09.309418 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 271/548] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:09.316041 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011485bb-dd53-4619-8612-27c1fb8ec7f2 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:09.320310 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfad815-f617-4ea2-9653-06841c8cfc28 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:09.392476 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:09.456615 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfad815-f617-4ea2-9653-06841c8cfc28 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:09.500214 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011485bb-dd53-4619-8612-27c1fb8ec7f2 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:09.515872 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:09.525101 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56419ba-96e5-4f88-9151-ddeff7e1a48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:09.526190 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56419ba-96e5-4f88-9151-ddeff7e1a48d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:09.526615 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56419ba-96e5-4f88-9151-ddeff7e1a48d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:09.528265 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56419ba-96e5-4f88-9151-ddeff7e1a48d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:09.540362 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56419ba-96e5-4f88-9151-ddeff7e1a48d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:09.548320 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-b56419ba-96e5-4f88-9151-ddeff7e1a48d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:09.548320 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:09.548320 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:09.548320 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:09.548320 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:09.548320 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:09.548320 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:09.548320 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:09.548320 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:09.548320 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:09.548320 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:09.548320 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:09.548320 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:09.548320 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:09.548320 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:09.548320 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:09.549729 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:09.549729 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:09.549729 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:09.549729 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:09.549729 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:09.549729 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:09.549729 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:09.549729 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:09.549729 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:09.549729 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:09.549729 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:09.549729 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:09.549729 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:09.549729 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:09.549729 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:09.549729 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:09.550630 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:09.550630 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:09.550630 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:09.550630 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:09.550630 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:09.550630 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:09.550630 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:09.550630 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:09.550630 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:09.550630 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:09.550630 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:09.550630 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:09.553565 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 272/549] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 23 16:54:09 2022] GET /identity/v3/roles/service => generated 85 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:09.563954 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56419ba-96e5-4f88-9151-ddeff7e1a48d None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:09.570070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01fe2b-e374-4370-8e71-98a5ba27f6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:09.571096 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01fe2b-e374-4370-8e71-98a5ba27f6a1 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:09.571585 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01fe2b-e374-4370-8e71-98a5ba27f6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:09.572053 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac01fe2b-e374-4370-8e71-98a5ba27f6a1 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:09.572980 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01fe2b-e374-4370-8e71-98a5ba27f6a1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:09.573481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01fe2b-e374-4370-8e71-98a5ba27f6a1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:09.575117 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac01fe2b-e374-4370-8e71-98a5ba27f6a1 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:09.589127 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 278/550] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:09.596015 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01fe2b-e374-4370-8e71-98a5ba27f6a1 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:09.720393 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56419ba-96e5-4f88-9151-ddeff7e1a48d None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:09.766136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9851c-b225-4f0e-bce9-e9ce0a594601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:09.767238 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9851c-b225-4f0e-bce9-e9ce0a594601 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:09.767715 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9851c-b225-4f0e-bce9-e9ce0a594601 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:09.768176 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9851c-b225-4f0e-bce9-e9ce0a594601 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:09.769126 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9851c-b225-4f0e-bce9-e9ce0a594601 None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:09.769625 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9851c-b225-4f0e-bce9-e9ce0a594601 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:09.772028 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e9851c-b225-4f0e-bce9-e9ce0a594601 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:09.781120 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 273/551] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 23 16:54:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:09.787202 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9851c-b225-4f0e-bce9-e9ce0a594601 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:09.801841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:09.819625 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac01fe2b-e374-4370-8e71-98a5ba27f6a1 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:09.866204 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e12d40-b51a-46e9-9847-3723fbfe7dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:09.869105 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e12d40-b51a-46e9-9847-3723fbfe7dee None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:09.869619 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e12d40-b51a-46e9-9847-3723fbfe7dee None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:09.870080 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e12d40-b51a-46e9-9847-3723fbfe7dee None admin] PATH_INFO: `/v3/users/swift` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:09.888170 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e12d40-b51a-46e9-9847-3723fbfe7dee None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:09.890477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e12d40-b51a-46e9-9847-3723fbfe7dee None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:09.917969 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:09.928099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-b1e12d40-b51a-46e9-9847-3723fbfe7dee None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 16:54:09.928099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:09.928099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:09.928099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:09.928099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:09.928099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:09.928099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:09.928099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:09.928099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:09.928099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:09.928099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:09.928099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:09.928099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:09.928099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:09.928099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:09.928099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:09.929115 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:09.929115 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:09.929115 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:09.929115 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:09.929115 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:09.929115 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:09.929115 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:09.929115 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:09.929115 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:09.929115 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:09.929115 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:09.929115 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:09.929115 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:09.929115 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:09.929115 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:09.929115 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:09.929115 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:09.929115 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:09.930092 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:09.930092 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:09.930092 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:09.930092 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:09.930092 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:09.930092 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:09.930092 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:09.930092 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:09.930092 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 16:54:09.930092 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:09.934844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 279/552] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:54:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:09.939647 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e12d40-b51a-46e9-9847-3723fbfe7dee None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:10.035213 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9851c-b225-4f0e-bce9-e9ce0a594601 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:10.060835 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e12d40-b51a-46e9-9847-3723fbfe7dee None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:10.068754 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d990ad96-6bed-4c19-9cc4-3befb6b85da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:10.069564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990ad96-6bed-4c19-9cc4-3befb6b85da9 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:10.070174 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990ad96-6bed-4c19-9cc4-3befb6b85da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:10.070174 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d990ad96-6bed-4c19-9cc4-3befb6b85da9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:10.080967 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d990ad96-6bed-4c19-9cc4-3befb6b85da9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:10.082558 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d990ad96-6bed-4c19-9cc4-3befb6b85da9 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:10.100860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-d990ad96-6bed-4c19-9cc4-3befb6b85da9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:10.100860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:10.100860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:10.100860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:10.100860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:10.100860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:10.100860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:10.100860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:10.100860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:10.100860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:10.100860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:10.100860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:10.100860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:10.100860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:10.100860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:10.101988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:10.101988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:10.101988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:10.101988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:10.101988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:10.101988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:10.101988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:10.101988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:10.101988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:10.101988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:10.101988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:10.101988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:10.101988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:10.101988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:10.101988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:10.101988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:10.101988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:10.103225 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:10.103225 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:10.103225 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:10.103225 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:10.103225 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 274/553] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:10.104010 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74823ef8-1c19-4355-90ac-d688d3cbb06b None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:10.104313 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74823ef8-1c19-4355-90ac-d688d3cbb06b None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:10.104568 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74823ef8-1c19-4355-90ac-d688d3cbb06b None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:10.106045 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 275/554] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:10.106656 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a8df81-3c82-4d0e-9d53-2622070b0315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:10.107647 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8df81-3c82-4d0e-9d53-2622070b0315 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:10.108067 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8df81-3c82-4d0e-9d53-2622070b0315 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:10.108476 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a8df81-3c82-4d0e-9d53-2622070b0315 None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:10.109550 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a8df81-3c82-4d0e-9d53-2622070b0315 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:10.110349 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a8df81-3c82-4d0e-9d53-2622070b0315 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:10.111245 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74823ef8-1c19-4355-90ac-d688d3cbb06b None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:10.112312 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a8df81-3c82-4d0e-9d53-2622070b0315 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:10.138797 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 280/555] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 23 16:54:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:10.140822 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01e4dd-692a-404e-8c41-893c52b24a04 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:10.141601 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01e4dd-692a-404e-8c41-893c52b24a04 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:10.142036 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae01e4dd-692a-404e-8c41-893c52b24a04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:10.164450 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-ae01e4dd-692a-404e-8c41-893c52b24a04 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:10.166098 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-ae01e4dd-692a-404e-8c41-893c52b24a04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:10.168747 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:10.227385 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74823ef8-1c19-4355-90ac-d688d3cbb06b None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:10.238879 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:10.261070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 281/556] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:54:10.262002 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01e4dd-692a-404e-8c41-893c52b24a04 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:10.284882 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc8cd4-6312-42d4-8c48-04326c0c9d6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:10.286480 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc8cd4-6312-42d4-8c48-04326c0c9d6d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:10.287258 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc8cd4-6312-42d4-8c48-04326c0c9d6d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:10.288579 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-facc8cd4-6312-42d4-8c48-04326c0c9d6d None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:10.291113 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc8cd4-6312-42d4-8c48-04326c0c9d6d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:10.291572 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc8cd4-6312-42d4-8c48-04326c0c9d6d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:10.293518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-facc8cd4-6312-42d4-8c48-04326c0c9d6d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:10.311409 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 276/557] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:10.312321 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc8cd4-6312-42d4-8c48-04326c0c9d6d None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:10.387110 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae01e4dd-692a-404e-8c41-893c52b24a04 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:10.417398 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-facc8cd4-6312-42d4-8c48-04326c0c9d6d None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:10.420119 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf62d1-93e6-4792-a7a3-e4d0d72ca426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:10.420780 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf62d1-93e6-4792-a7a3-e4d0d72ca426 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:10.421065 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf62d1-93e6-4792-a7a3-e4d0d72ca426 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:10.421347 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46bf62d1-93e6-4792-a7a3-e4d0d72ca426 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:10.429692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:10.456995 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adba231a-c44c-43a2-9d85-0c92f00d5955 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:10.457928 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba231a-c44c-43a2-9d85-0c92f00d5955 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:10.458210 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba231a-c44c-43a2-9d85-0c92f00d5955 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:10.458487 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adba231a-c44c-43a2-9d85-0c92f00d5955 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:10.474252 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adba231a-c44c-43a2-9d85-0c92f00d5955 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:10.482151 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-adba231a-c44c-43a2-9d85-0c92f00d5955 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:10.482151 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:10.482151 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:10.482151 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:10.482151 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:10.482151 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:10.482151 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:10.482151 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:10.482151 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:10.482151 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:10.482151 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:10.482151 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:10.482151 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:10.482151 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:10.482151 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:10.482151 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:10.488698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:10.488698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:10.488698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:10.488698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:10.488698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:10.488698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:10.488698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:10.488698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:10.488698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:10.488698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:10.488698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:10.488698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:10.488698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:10.488698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:10.488698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:10.488698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:10.489521 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:10.489521 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:10.489521 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:10.489521 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:10.489521 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:10.489521 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:10.489521 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:10.489521 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:10.489521 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:10.489521 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:10.489521 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:10.489521 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:10.489521 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 277/558] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 23 16:54:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:10.489521 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf4ba2-13a9-41c9-a67a-bf73108864c7 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:10.490260 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf4ba2-13a9-41c9-a67a-bf73108864c7 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:10.490260 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8cf4ba2-13a9-41c9-a67a-bf73108864c7 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:10.491046 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 278/559] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:10 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:10.491857 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf4ba2-13a9-41c9-a67a-bf73108864c7 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:10.496008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf62d1-93e6-4792-a7a3-e4d0d72ca426 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:10.498352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46bf62d1-93e6-4792-a7a3-e4d0d72ca426 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:10.521046 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:10.607137 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8cf4ba2-13a9-41c9-a67a-bf73108864c7 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:10.612336 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 282/560] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:54:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:54:10.612336 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf62d1-93e6-4792-a7a3-e4d0d72ca426 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:10.647529 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769c7951-731a-473a-871d-dce273e0251b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:10.648319 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c7951-731a-473a-871d-dce273e0251b None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:10.648585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c7951-731a-473a-871d-dce273e0251b None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:10.648856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769c7951-731a-473a-871d-dce273e0251b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:10.655046 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:10.656927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769c7951-731a-473a-871d-dce273e0251b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:10.658424 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769c7951-731a-473a-871d-dce273e0251b None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:10.673343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-769c7951-731a-473a-871d-dce273e0251b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 16:54:10.673343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:10.673343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:10.673343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:10.673343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:10.673343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:10.673343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:10.673343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:10.673343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:10.673343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:10.673343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:10.673343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:10.673343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:10.673343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:10.673343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:10.673343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:10.674412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:10.674412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:10.674412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:10.674412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:10.674412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:10.674412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:10.674412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:10.674412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:10.674412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:10.674412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:10.674412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:10.674412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:10.674412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:10.674412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:10.674412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:10.674412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:10.674412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:10.674412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:10.675363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:10.675363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:10.675363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:10.675363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:10.675363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:10.675363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:10.675363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:10.675363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:10.675363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 16:54:10.675363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:10.675363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 279/561] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 23 16:54:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:10.676034 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913dbaa2-9330-46fd-a929-0a319a297548 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:10.676101 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913dbaa2-9330-46fd-a929-0a319a297548 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:10.676977 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913dbaa2-9330-46fd-a929-0a319a297548 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:10.696800 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-913dbaa2-9330-46fd-a929-0a319a297548 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:10.698390 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-913dbaa2-9330-46fd-a929-0a319a297548 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:10.723279 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46bf62d1-93e6-4792-a7a3-e4d0d72ca426 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:10.764307 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9632d-0366-47be-96b5-134c44bb7b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:10.765380 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9632d-0366-47be-96b5-134c44bb7b21 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:10.765865 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9632d-0366-47be-96b5-134c44bb7b21 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:10.766346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b9632d-0366-47be-96b5-134c44bb7b21 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:10.767733 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b9632d-0366-47be-96b5-134c44bb7b21 None admin] RBAC: Adding query filter params (name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:10.768276 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b9632d-0366-47be-96b5-134c44bb7b21 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:10.770258 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b9632d-0366-47be-96b5-134c44bb7b21 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:10.783687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 280/563] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:54:10.783687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913dbaa2-9330-46fd-a929-0a319a297548 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:10.791001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 283/563] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 23 16:54:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:10.791971 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9632d-0366-47be-96b5-134c44bb7b21 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:10.799261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:10.885321 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913dbaa2-9330-46fd-a929-0a319a297548 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:10.904125 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:10.907954 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b9632d-0366-47be-96b5-134c44bb7b21 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:10.922268 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0ea9e-3e37-460a-8c91-bb17c64ee8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:10.923106 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0ea9e-3e37-460a-8c91-bb17c64ee8df None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:10.923389 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0ea9e-3e37-460a-8c91-bb17c64ee8df None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:10.923661 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a0ea9e-3e37-460a-8c91-bb17c64ee8df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:10.934569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a0ea9e-3e37-460a-8c91-bb17c64ee8df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:10.936080 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a0ea9e-3e37-460a-8c91-bb17c64ee8df None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:10.943941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdb7c6-6cbd-463a-b9b8-99506ec88082 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:10.944618 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdb7c6-6cbd-463a-b9b8-99506ec88082 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:10.944900 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdb7c6-6cbd-463a-b9b8-99506ec88082 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:10.945169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cdb7c6-6cbd-463a-b9b8-99506ec88082 None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:10.946007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cdb7c6-6cbd-463a-b9b8-99506ec88082 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:10.946314 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cdb7c6-6cbd-463a-b9b8-99506ec88082 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:10.947766 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cdb7c6-6cbd-463a-b9b8-99506ec88082 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:10.953344 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-61a0ea9e-3e37-460a-8c91-bb17c64ee8df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:10.953344 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:10.953344 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:10.953344 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:10.953344 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:10.953344 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:10.953344 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:10.953344 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:10.953344 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:10.953344 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:10.953344 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:10.953344 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:10.953344 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:10.953344 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:10.953344 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:10.954347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:10.954347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:10.954347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:10.954347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:10.954347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:10.954347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:10.954347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:10.954347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:10.954347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:10.954347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:10.954347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:10.954347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:10.954347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:10.954347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:10.954347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:10.954347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:10.954347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:10.955267 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:10.955267 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:10.955267 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:10.955267 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:10.955267 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 281/564] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:10.955559 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0ea9e-3e37-460a-8c91-bb17c64ee8df None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:10.968751 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 284/565] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 23 16:54:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:10.968751 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdb7c6-6cbd-463a-b9b8-99506ec88082 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:11.024037 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:11.057560 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a0ea9e-3e37-460a-8c91-bb17c64ee8df None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:11.067854 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cdb7c6-6cbd-463a-b9b8-99506ec88082 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:11.092176 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d3d3e-769e-43c7-850b-4d719ec18f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:11.092966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d3d3e-769e-43c7-850b-4d719ec18f12 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:11.093246 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d3d3e-769e-43c7-850b-4d719ec18f12 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:11.093516 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4d3d3e-769e-43c7-850b-4d719ec18f12 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:11.102024 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4ac4bb-9530-4b6f-937a-d404a3a7ff91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:11.102825 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ac4bb-9530-4b6f-937a-d404a3a7ff91 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:11.103146 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ac4bb-9530-4b6f-937a-d404a3a7ff91 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:11.103417 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4ac4bb-9530-4b6f-937a-d404a3a7ff91 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:11.103516 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d3d3e-769e-43c7-850b-4d719ec18f12 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:11.104963 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee4d3d3e-769e-43c7-850b-4d719ec18f12 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:11.113939 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4ac4bb-9530-4b6f-937a-d404a3a7ff91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:11.121736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-1f4ac4bb-9530-4b6f-937a-d404a3a7ff91 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:11.121736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:11.121736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:11.121736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:11.121736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:11.121736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:11.121736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:11.121736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:11.121736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:11.121736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:11.121736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:11.121736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:11.121736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:11.121736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:11.121736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:11.121736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:11.122662 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:11.122662 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:11.122662 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:11.122662 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:11.122662 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:11.122662 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:11.122662 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:11.122662 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:11.122662 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:11.122662 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:11.122662 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:11.122662 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:11.122662 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:11.122662 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:11.122662 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:11.122662 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:11.123744 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:11.123744 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:11.123744 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:11.123744 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:11.123744 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:11.123744 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:11.123744 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:11.123744 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:11.123744 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:11.123744 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:11.123744 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:11.123744 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:11.124371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-ee4d3d3e-769e-43c7-850b-4d719ec18f12 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:11.124371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:11.124371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:11.124371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:11.124371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:11.124371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:11.124371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:11.124371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:11.124371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:11.124371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:11.124371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:11.124371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:11.124371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:11.124371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:11.124371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:11.125129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:11.125129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:11.125129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:11.125129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:11.125129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:11.125129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:11.125129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:11.125129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:11.125129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:11.125129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:11.125129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:11.125129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:11.125129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:11.125129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:11.125129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:11.125129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:11.125129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:11.125998 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:11.125998 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:11.125998 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:11.125998 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:11.126209 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 285/566] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 23 16:54:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:11.126284 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 282/567] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:11.126284 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d3d3e-769e-43c7-850b-4d719ec18f12 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:11.128126 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a903b2-87e1-44ac-bfac-57265dfd5dfd None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:11.128408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a903b2-87e1-44ac-bfac-57265dfd5dfd None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:11.128678 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a903b2-87e1-44ac-bfac-57265dfd5dfd None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:11.130085 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 286/568] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:11 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:11.130801 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a903b2-87e1-44ac-bfac-57265dfd5dfd None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:11.151067 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:11.230020 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4d3d3e-769e-43c7-850b-4d719ec18f12 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:11.234684 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a903b2-87e1-44ac-bfac-57265dfd5dfd None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:11.249475 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:11.263973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1e8f8-d148-44b9-bc8f-5809a488241c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:11.264630 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1e8f8-d148-44b9-bc8f-5809a488241c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:11.264904 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1e8f8-d148-44b9-bc8f-5809a488241c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:11.265172 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b1e8f8-d148-44b9-bc8f-5809a488241c None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:11.265885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b1e8f8-d148-44b9-bc8f-5809a488241c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:11.266187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b1e8f8-d148-44b9-bc8f-5809a488241c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:11.267606 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0b1e8f8-d148-44b9-bc8f-5809a488241c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:11.269190 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80659c1d-6223-4268-addf-20d9bb151706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:11.270559 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80659c1d-6223-4268-addf-20d9bb151706 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:11.270841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80659c1d-6223-4268-addf-20d9bb151706 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:11.271146 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80659c1d-6223-4268-addf-20d9bb151706 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:11.278827 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 283/569] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:11.279654 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1e8f8-d148-44b9-bc8f-5809a488241c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:11.281523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80659c1d-6223-4268-addf-20d9bb151706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:11.283090 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80659c1d-6223-4268-addf-20d9bb151706 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:11.302095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-80659c1d-6223-4268-addf-20d9bb151706 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:11.302095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:11.302095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:11.302095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:11.302095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:11.302095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:11.302095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:11.302095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:11.302095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:11.302095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:11.302095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:11.302095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:11.302095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:11.302095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:11.302095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:11.303114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:11.303114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:11.303114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:11.303114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:11.303114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:11.303114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:11.303114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:11.303114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:11.303114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:11.303114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:11.303114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:11.303114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:11.303114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:11.303114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:11.303114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:11.303114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:11.303114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:11.304371 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:11.304371 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:11.304371 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:11.304371 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:11.304371 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 287/570] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:11.304371 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80659c1d-6223-4268-addf-20d9bb151706 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:11.372213 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:11.382700 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0b1e8f8-d148-44b9-bc8f-5809a488241c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:11.404728 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80659c1d-6223-4268-addf-20d9bb151706 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:11.416174 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d231dc18-cea0-43dd-8e0f-7c62225344b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:11.416850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231dc18-cea0-43dd-8e0f-7c62225344b7 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:11.417129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231dc18-cea0-43dd-8e0f-7c62225344b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:11.417407 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d231dc18-cea0-43dd-8e0f-7c62225344b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:11.418125 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231dc18-cea0-43dd-8e0f-7c62225344b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:11.418429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231dc18-cea0-43dd-8e0f-7c62225344b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:11.419851 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d231dc18-cea0-43dd-8e0f-7c62225344b7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:11.432664 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 284/571] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:11.438417 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1c141-99d3-4b37-8044-9e81ca417af1 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:11.438710 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1c141-99d3-4b37-8044-9e81ca417af1 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:11.439068 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e1c141-99d3-4b37-8044-9e81ca417af1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:11.440147 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde67991-2ed8-4fea-97df-b0698caa5149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:11.440814 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde67991-2ed8-4fea-97df-b0698caa5149 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:11.441094 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde67991-2ed8-4fea-97df-b0698caa5149 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:11.441367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde67991-2ed8-4fea-97df-b0698caa5149 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:11.442758 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde67991-2ed8-4fea-97df-b0698caa5149 None admin] RBAC: Adding query filter params (name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:11.443102 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde67991-2ed8-4fea-97df-b0698caa5149 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:11.444516 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde67991-2ed8-4fea-97df-b0698caa5149 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:11.452848 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 288/572] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 23 16:54:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:11.453496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde67991-2ed8-4fea-97df-b0698caa5149 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:11.455273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-96e1c141-99d3-4b37-8044-9e81ca417af1 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:11.456499 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-96e1c141-99d3-4b37-8044-9e81ca417af1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:11.494681 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:11.532449 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 285/573] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:54:11.533287 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1c141-99d3-4b37-8044-9e81ca417af1 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:11.564434 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde67991-2ed8-4fea-97df-b0698caa5149 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:11.594589 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:11.600624 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec57c9-f3ac-4719-8dbe-778e743c9c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:11.601487 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec57c9-f3ac-4719-8dbe-778e743c9c61 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:11.601763 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec57c9-f3ac-4719-8dbe-778e743c9c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:11.602036 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bec57c9-f3ac-4719-8dbe-778e743c9c61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:11.608808 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec57c9-f3ac-4719-8dbe-778e743c9c61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:11.610248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bec57c9-f3ac-4719-8dbe-778e743c9c61 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:11.625463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-6bec57c9-f3ac-4719-8dbe-778e743c9c61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:11.625463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:11.625463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:11.625463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:11.625463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:11.625463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:11.625463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:11.625463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:11.625463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:11.625463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:11.625463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:11.625463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:11.625463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:11.625463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:11.625463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:11.627021 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:11.627021 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:11.627021 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:11.627021 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:11.627021 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:11.627021 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:11.627021 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:11.627021 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:11.627021 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:11.627021 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:11.627021 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:11.627021 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:11.627021 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:11.627021 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:11.627021 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:11.627021 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:11.627021 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:11.628189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:11.628189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:11.628189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:11.628189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:11.628189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:11.628189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:11.628189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:11.628189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:11.628189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 289/574] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:11.628189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec57c9-f3ac-4719-8dbe-778e743c9c61 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:11.634425 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e1c141-99d3-4b37-8044-9e81ca417af1 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:11.667159 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7b3f2-0b77-4902-a940-10fa854cf1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:11.667743 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7b3f2-0b77-4902-a940-10fa854cf1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:11.668028 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7b3f2-0b77-4902-a940-10fa854cf1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:11.668297 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f7b3f2-0b77-4902-a940-10fa854cf1a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:11.669016 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7b3f2-0b77-4902-a940-10fa854cf1a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:11.669317 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7b3f2-0b77-4902-a940-10fa854cf1a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:11.670705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f7b3f2-0b77-4902-a940-10fa854cf1a5 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:11.681685 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 286/575] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:11.682471 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7b3f2-0b77-4902-a940-10fa854cf1a5 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:11.715243 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:11.732475 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bec57c9-f3ac-4719-8dbe-778e743c9c61 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:11.766976 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62b65b-fc3b-41f0-8c68-70a81e192d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:11.767781 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62b65b-fc3b-41f0-8c68-70a81e192d23 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:11.768063 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62b65b-fc3b-41f0-8c68-70a81e192d23 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:11.768338 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62b65b-fc3b-41f0-8c68-70a81e192d23 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:11.775872 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62b65b-fc3b-41f0-8c68-70a81e192d23 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:11.777328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a62b65b-fc3b-41f0-8c68-70a81e192d23 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:11.784143 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f7b3f2-0b77-4902-a940-10fa854cf1a5 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:11.792141 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-8a62b65b-fc3b-41f0-8c68-70a81e192d23 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 23 16:54:11.792141 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:11.792141 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:11.792141 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:11.792141 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:11.792141 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:11.792141 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:11.792141 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:11.792141 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:11.792141 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:11.792141 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:11.792141 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:11.792141 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:11.792141 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:11.792141 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:11.792141 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:11.793088 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:11.793088 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:11.793088 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:11.793088 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:11.793088 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:11.793088 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:11.793088 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:11.793088 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:11.793088 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:11.793088 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:11.793088 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:11.793088 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:11.793088 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:11.793088 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:11.793088 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:11.793088 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:11.793088 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:11.793088 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:11.794038 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:11.794038 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:11.794038 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:11.794038 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:11.794038 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:11.794038 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:11.794038 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:11.794038 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:11.794038 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 23 16:54:11.794038 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:11.794038 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 290/576] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:54:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:11.794038 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62b65b-fc3b-41f0-8c68-70a81e192d23 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:11.822572 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36bbc2-fbe6-4e30-beb6-8bf73559f30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:11.823537 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36bbc2-fbe6-4e30-beb6-8bf73559f30d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:11.823842 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36bbc2-fbe6-4e30-beb6-8bf73559f30d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:11.824111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d36bbc2-fbe6-4e30-beb6-8bf73559f30d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:11.834621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36bbc2-fbe6-4e30-beb6-8bf73559f30d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:11.836112 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d36bbc2-fbe6-4e30-beb6-8bf73559f30d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:11.845699 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:11.852829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-7d36bbc2-fbe6-4e30-beb6-8bf73559f30d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:11.852829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:11.852829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:11.852829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:11.852829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:11.852829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:11.852829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:11.852829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:11.852829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:11.852829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:11.852829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:11.852829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:11.852829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:11.852829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:11.852829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:11.853702 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:11.853702 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:11.853702 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:11.853702 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:11.853702 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:11.853702 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:11.853702 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:11.853702 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:11.853702 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:11.853702 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:11.853702 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:11.853702 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:11.853702 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:11.853702 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:11.853702 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:11.853702 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:11.853702 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:11.854572 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:11.854572 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:11.854572 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:11.854572 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:11.854572 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 287/577] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:11.854854 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36bbc2-fbe6-4e30-beb6-8bf73559f30d None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:11.897357 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62b65b-fc3b-41f0-8c68-70a81e192d23 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:11.930985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3690644-a04b-4ecc-afb6-d95067fa4765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:11.931728 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3690644-a04b-4ecc-afb6-d95067fa4765 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:11.932023 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3690644-a04b-4ecc-afb6-d95067fa4765 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:11.932290 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3690644-a04b-4ecc-afb6-d95067fa4765 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:11.937412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:11.942833 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3690644-a04b-4ecc-afb6-d95067fa4765 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:11.950591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-c3690644-a04b-4ecc-afb6-d95067fa4765 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:11.950591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:11.950591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:11.950591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:11.950591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:11.950591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:11.950591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:11.950591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:11.950591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:11.950591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:11.950591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:11.950591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:11.950591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:11.950591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:11.950591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:11.950591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:11.951701 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:11.951701 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:11.951701 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:11.951701 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:11.951701 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:11.951701 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:11.951701 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:11.951701 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:11.951701 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:11.951701 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:11.951701 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:11.951701 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:11.951701 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:11.951701 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:11.951701 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:11.951701 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:11.952552 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:11.952552 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:11.952552 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:11.952552 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:11.952552 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:11.952552 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:11.952552 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:11.952552 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:11.952552 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:11.952552 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:11.952552 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:11.952552 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:11.952552 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 291/578] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 23 16:54:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:11.952552 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3690644-a04b-4ecc-afb6-d95067fa4765 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:11.960421 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d36bbc2-fbe6-4e30-beb6-8bf73559f30d None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:11.994431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1dcafa-f425-44f9-9d91-be83c9a98e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:11.995129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1dcafa-f425-44f9-9d91-be83c9a98e33 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:11.995414 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1dcafa-f425-44f9-9d91-be83c9a98e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:11.995692 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1dcafa-f425-44f9-9d91-be83c9a98e33 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:11.996478 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1dcafa-f425-44f9-9d91-be83c9a98e33 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:11.996789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1dcafa-f425-44f9-9d91-be83c9a98e33 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:11.998222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1dcafa-f425-44f9-9d91-be83c9a98e33 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:12.010406 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 288/579] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:12.011407 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1dcafa-f425-44f9-9d91-be83c9a98e33 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:12.056268 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3690644-a04b-4ecc-afb6-d95067fa4765 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:12.068725 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:12.090756 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113254f-81e6-48bf-9856-49439ec7cc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:12.091583 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113254f-81e6-48bf-9856-49439ec7cc59 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:12.091852 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113254f-81e6-48bf-9856-49439ec7cc59 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:12.092193 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0113254f-81e6-48bf-9856-49439ec7cc59 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:12.099182 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113254f-81e6-48bf-9856-49439ec7cc59 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:12.100610 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0113254f-81e6-48bf-9856-49439ec7cc59 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:12.111085 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1dcafa-f425-44f9-9d91-be83c9a98e33 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:12.114588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-0113254f-81e6-48bf-9856-49439ec7cc59 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:12.114588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:12.114588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:12.114588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:12.114588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:12.114588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:12.114588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:12.114588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:12.114588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:12.114588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:12.114588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:12.114588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:12.114588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:12.114588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:12.114588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:12.115614 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:12.115614 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:12.115614 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:12.115614 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:12.115614 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:12.115614 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:12.115614 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:12.115614 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:12.115614 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:12.115614 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:12.115614 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:12.115614 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:12.115614 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:12.115614 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:12.115614 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:12.115614 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:12.115614 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:12.116591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:12.116591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:12.116591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:12.116591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:12.116591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:12.116591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:12.116591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:12.116591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:12.116591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 292/580] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:12.116591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113254f-81e6-48bf-9856-49439ec7cc59 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:12.146212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92d517-5919-46f0-9ec4-77295e7bf6ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:12.147061 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92d517-5919-46f0-9ec4-77295e7bf6ff None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:12.147444 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92d517-5919-46f0-9ec4-77295e7bf6ff None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:12.147811 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e92d517-5919-46f0-9ec4-77295e7bf6ff None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:12.148741 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92d517-5919-46f0-9ec4-77295e7bf6ff None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:12.149148 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92d517-5919-46f0-9ec4-77295e7bf6ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:12.150656 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e92d517-5919-46f0-9ec4-77295e7bf6ff None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:12.165329 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:12.169361 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 289/581] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 23 16:54:12 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:12.170172 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92d517-5919-46f0-9ec4-77295e7bf6ff None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:12.222805 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0113254f-81e6-48bf-9856-49439ec7cc59 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:12.256934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce7953-604b-46af-ace0-48ee3ffb70ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:12.257608 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce7953-604b-46af-ace0-48ee3ffb70ad None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:12.257898 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce7953-604b-46af-ace0-48ee3ffb70ad None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:12.258162 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce7953-604b-46af-ace0-48ee3ffb70ad None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:12.259244 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcce7953-604b-46af-ace0-48ee3ffb70ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:12.259547 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcce7953-604b-46af-ace0-48ee3ffb70ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:12.260929 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcce7953-604b-46af-ace0-48ee3ffb70ad None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:12.271999 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 293/582] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:12.272777 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce7953-604b-46af-ace0-48ee3ffb70ad None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:12.272877 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e92d517-5919-46f0-9ec4-77295e7bf6ff None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:12.304235 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:12.307367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa1deb-f6d9-4154-af1d-d7401e0e47ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:12.308043 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa1deb-f6d9-4154-af1d-d7401e0e47ab None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:12.308322 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa1deb-f6d9-4154-af1d-d7401e0e47ab None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:12.308602 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fa1deb-f6d9-4154-af1d-d7401e0e47ab None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:12.309343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa1deb-f6d9-4154-af1d-d7401e0e47ab None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:12.309645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa1deb-f6d9-4154-af1d-d7401e0e47ab None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:12.311049 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4fa1deb-f6d9-4154-af1d-d7401e0e47ab None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:12.319263 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 290/583] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 23 16:54:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:12.320236 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa1deb-f6d9-4154-af1d-d7401e0e47ab None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:12.377579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce7953-604b-46af-ace0-48ee3ffb70ad None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:12.381187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:12.411353 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef74c52-3b74-437d-a2fb-3dc52688d554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:12.412010 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef74c52-3b74-437d-a2fb-3dc52688d554 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:12.412288 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef74c52-3b74-437d-a2fb-3dc52688d554 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:12.412558 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef74c52-3b74-437d-a2fb-3dc52688d554 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:12.413379 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef74c52-3b74-437d-a2fb-3dc52688d554 None admin] RBAC: Adding query filter params (role.id=5a7214dc069f45969e19c44122b378a4, scope.project.id=0595667fb2e04a3ca4716f669d6698b0, user.id=6546217667054abb8eb07076995c1b04) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:12.413688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef74c52-3b74-437d-a2fb-3dc52688d554 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:12.415178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef74c52-3b74-437d-a2fb-3dc52688d554 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:12.420457 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 294/584] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:54:12 2022] GET /identity/v3/role_assignments?user.id=6546217667054abb8eb07076995c1b04&scope.project.id=0595667fb2e04a3ca4716f669d6698b0&role.id=5a7214dc069f45969e19c44122b378a4 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:12.420573 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fa1deb-f6d9-4154-af1d-d7401e0e47ab None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:12.421218 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef74c52-3b74-437d-a2fb-3dc52688d554 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:12.467623 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5997d0-1c94-40a6-9ce9-fe63651b332d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:12.468435 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5997d0-1c94-40a6-9ce9-fe63651b332d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:12.468783 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5997d0-1c94-40a6-9ce9-fe63651b332d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:12.469130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f5997d0-1c94-40a6-9ce9-fe63651b332d None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:12.470009 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5997d0-1c94-40a6-9ce9-fe63651b332d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:12.470391 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5997d0-1c94-40a6-9ce9-fe63651b332d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:12.471976 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f5997d0-1c94-40a6-9ce9-fe63651b332d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:12.490162 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 291/585] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:12.499111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5997d0-1c94-40a6-9ce9-fe63651b332d None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:12.536295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:12.543100 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef74c52-3b74-437d-a2fb-3dc52688d554 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:12.583384 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c367a-3581-498a-b011-a95f3ce45d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:12.584203 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c367a-3581-498a-b011-a95f3ce45d80 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:12.584488 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c367a-3581-498a-b011-a95f3ce45d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:12.584767 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6c367a-3581-498a-b011-a95f3ce45d80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:12.596317 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c367a-3581-498a-b011-a95f3ce45d80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:12.597943 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6c367a-3581-498a-b011-a95f3ce45d80 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:12.607075 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f5997d0-1c94-40a6-9ce9-fe63651b332d None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:12.615923 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-8a6c367a-3581-498a-b011-a95f3ce45d80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:12.615923 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:12.615923 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:12.615923 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:12.615923 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:12.615923 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:12.615923 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:12.615923 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:12.615923 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:12.615923 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:12.615923 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:12.615923 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:12.615923 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:12.615923 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:12.615923 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:12.616838 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:12.616838 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:12.616838 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:12.616838 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:12.616838 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:12.616838 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:12.616838 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:12.616838 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:12.616838 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:12.616838 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:12.616838 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:12.616838 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:12.616838 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:12.616838 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:12.616838 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:12.616838 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:12.616838 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:12.617742 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:12.617742 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:12.617742 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:12.617742 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:12.617742 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 295/586] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:12.618023 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c367a-3581-498a-b011-a95f3ce45d80 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:12.637697 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:12.650607 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea3065-670c-45fd-aa1d-dd0081e17fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:12.651461 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea3065-670c-45fd-aa1d-dd0081e17fbc None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:12.651785 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea3065-670c-45fd-aa1d-dd0081e17fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:12.652348 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bea3065-670c-45fd-aa1d-dd0081e17fbc None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:12.661408 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea3065-670c-45fd-aa1d-dd0081e17fbc None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:12.662876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bea3065-670c-45fd-aa1d-dd0081e17fbc None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:12.678179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-4bea3065-670c-45fd-aa1d-dd0081e17fbc None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 23 16:54:12.678179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:12.678179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:12.678179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:12.678179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:12.678179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:12.678179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:12.678179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:12.678179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:12.678179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:12.678179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:12.678179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:12.678179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:12.678179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:12.678179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:12.678179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:12.679169 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:12.679169 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:12.679169 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:12.679169 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:12.679169 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:12.679169 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:12.679169 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:12.679169 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:12.679169 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:12.679169 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:12.679169 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:12.679169 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:12.679169 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:12.679169 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:12.679169 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:12.679169 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:12.679169 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:12.679169 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:12.680110 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:12.680110 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:12.680110 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:12.680110 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:12.680110 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:12.680110 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:12.680110 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:12.680110 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:12.680110 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 23 16:54:12.680110 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:12.680110 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 292/587] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 23 16:54:12 2022] GET /identity/v3/users/neutron => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:12.680687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea3065-670c-45fd-aa1d-dd0081e17fbc None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:12.735579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6c367a-3581-498a-b011-a95f3ce45d80 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:12.773758 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9c913-8df6-4ef8-ba1d-9a8cbb61c38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:12.774559 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9c913-8df6-4ef8-ba1d-9a8cbb61c38d None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:12.774842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9c913-8df6-4ef8-ba1d-9a8cbb61c38d None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:12.775148 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a9c913-8df6-4ef8-ba1d-9a8cbb61c38d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:12.785258 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bea3065-670c-45fd-aa1d-dd0081e17fbc None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:12.786753 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a9c913-8df6-4ef8-ba1d-9a8cbb61c38d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:12.788239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a9c913-8df6-4ef8-ba1d-9a8cbb61c38d None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:12.797150 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:12.808333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-a1a9c913-8df6-4ef8-ba1d-9a8cbb61c38d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:12.808333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:12.808333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:12.808333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:12.808333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:12.808333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:12.808333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:12.808333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:12.808333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:12.808333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:12.808333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:12.808333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:12.808333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:12.808333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:12.808333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:12.809509 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:12.809509 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:12.809509 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:12.809509 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:12.809509 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:12.809509 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:12.809509 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:12.809509 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:12.809509 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:12.809509 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:12.809509 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:12.809509 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:12.809509 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:12.809509 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:12.809509 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:12.809509 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:12.809509 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:12.810606 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:12.810606 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:12.810606 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:12.810606 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:12.812978 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 296/588] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:12.818975 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9c913-8df6-4ef8-ba1d-9a8cbb61c38d None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:12.822683 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d314ad-c768-45ad-8fe7-c9cd9b11230c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:12.823647 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d314ad-c768-45ad-8fe7-c9cd9b11230c None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:12.823900 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d314ad-c768-45ad-8fe7-c9cd9b11230c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:12.824199 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d314ad-c768-45ad-8fe7-c9cd9b11230c None admin] PATH_INFO: `/v3/projects/0595667fb2e04a3ca4716f669d6698b0/users/225c7b385f12445fab86b3b5606b1da1/roles/5a7214dc069f45969e19c44122b378a4` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:12.841014 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d314ad-c768-45ad-8fe7-c9cd9b11230c None admin] RBAC: Authorizing `identity:create_grant(project_id=0595667fb2e04a3ca4716f669d6698b0, user_id=225c7b385f12445fab86b3b5606b1da1, role_id=5a7214dc069f45969e19c44122b378a4)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:12.843081 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d314ad-c768-45ad-8fe7-c9cd9b11230c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:12.859852 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 293/589] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:54:12 2022] PUT /identity/v3/projects/0595667fb2e04a3ca4716f669d6698b0/users/225c7b385f12445fab86b3b5606b1da1/roles/5a7214dc069f45969e19c44122b378a4 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:54:12.860905 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d314ad-c768-45ad-8fe7-c9cd9b11230c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:12.917281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:12.937973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a9c913-8df6-4ef8-ba1d-9a8cbb61c38d None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:12.992323 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054a7c4-b3e0-443b-bbcb-47e8679de0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:12.993337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054a7c4-b3e0-443b-bbcb-47e8679de0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:12.993717 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054a7c4-b3e0-443b-bbcb-47e8679de0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:12.994153 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5054a7c4-b3e0-443b-bbcb-47e8679de0a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:12.995418 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054a7c4-b3e0-443b-bbcb-47e8679de0a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:12.995831 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054a7c4-b3e0-443b-bbcb-47e8679de0a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:12.997415 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5054a7c4-b3e0-443b-bbcb-47e8679de0a3 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:13.014449 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d314ad-c768-45ad-8fe7-c9cd9b11230c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:13.020828 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 297/590] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:13.020828 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054a7c4-b3e0-443b-bbcb-47e8679de0a3 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:13.063089 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0fa18c-f0d9-4a80-9a8c-25f770d189e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:13.064038 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fa18c-f0d9-4a80-9a8c-25f770d189e4 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:13.064416 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fa18c-f0d9-4a80-9a8c-25f770d189e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:13.064784 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0fa18c-f0d9-4a80-9a8c-25f770d189e4 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:13.065767 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0fa18c-f0d9-4a80-9a8c-25f770d189e4 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:13.066171 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0fa18c-f0d9-4a80-9a8c-25f770d189e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:13.067860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0fa18c-f0d9-4a80-9a8c-25f770d189e4 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:13.090939 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 294/591] 10.222.0.13 () {56 vars in 1139 bytes} [Mon May 23 16:54:12 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:13.091766 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0fa18c-f0d9-4a80-9a8c-25f770d189e4 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:13.094846 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:13.149828 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5054a7c4-b3e0-443b-bbcb-47e8679de0a3 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:13.194530 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4274a-d971-4915-9b05-476f74879940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:13.195244 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4274a-d971-4915-9b05-476f74879940 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:13.195531 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4274a-d971-4915-9b05-476f74879940 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:13.195810 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f4274a-d971-4915-9b05-476f74879940 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:13.196702 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f4274a-d971-4915-9b05-476f74879940 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:13.197018 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f4274a-d971-4915-9b05-476f74879940 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:13.199057 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f4274a-d971-4915-9b05-476f74879940 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:13.203693 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0fa18c-f0d9-4a80-9a8c-25f770d189e4 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:13.218438 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 298/592] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:13.219381 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4274a-d971-4915-9b05-476f74879940 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:13.244080 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1da646-0cdc-404b-8f06-537cf8b41856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:13.245278 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1da646-0cdc-404b-8f06-537cf8b41856 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:13.245748 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1da646-0cdc-404b-8f06-537cf8b41856 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:13.246253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1da646-0cdc-404b-8f06-537cf8b41856 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:13.247226 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:13.255475 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1da646-0cdc-404b-8f06-537cf8b41856 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:13.258608 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1da646-0cdc-404b-8f06-537cf8b41856 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:13.276626 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-4e1da646-0cdc-404b-8f06-537cf8b41856 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:13.276626 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:13.276626 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:13.276626 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:13.276626 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:13.276626 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:13.276626 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:13.276626 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:13.276626 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:13.276626 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:13.276626 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:13.276626 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:13.276626 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:13.276626 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:13.276626 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:13.277534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:13.277534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:13.277534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:13.277534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:13.277534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:13.277534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:13.277534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:13.277534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:13.277534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:13.277534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:13.277534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:13.277534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:13.277534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:13.277534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:13.277534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:13.277534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:13.277534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:13.278460 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:13.278460 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:13.278460 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:13.278460 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:13.278460 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:13.278460 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:13.278460 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:13.278460 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:13.285223 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 295/593] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:13.285223 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1da646-0cdc-404b-8f06-537cf8b41856 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:13.350670 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f4274a-d971-4915-9b05-476f74879940 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:13.372463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:13.389778 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac39501-da86-4059-81a3-e5a40ebc0291 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:13.391044 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac39501-da86-4059-81a3-e5a40ebc0291 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:13.391546 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac39501-da86-4059-81a3-e5a40ebc0291 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:13.392052 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac39501-da86-4059-81a3-e5a40ebc0291 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:13.400577 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1da646-0cdc-404b-8f06-537cf8b41856 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:13.404922 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac39501-da86-4059-81a3-e5a40ebc0291 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:13.406592 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac39501-da86-4059-81a3-e5a40ebc0291 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:13.425345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-aac39501-da86-4059-81a3-e5a40ebc0291 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:13.425345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:13.425345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:13.425345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:13.425345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:13.425345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:13.425345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:13.425345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:13.425345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:13.425345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:13.425345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:13.425345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:13.425345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:13.425345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:13.425345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:13.426604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:13.426604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:13.426604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:13.426604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:13.426604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:13.426604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:13.426604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:13.426604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:13.426604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:13.426604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:13.426604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:13.426604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:13.426604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:13.426604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:13.426604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:13.426604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:13.426604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:13.427516 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:13.427516 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:13.427516 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:13.427516 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:13.429946 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 299/594] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:13.431216 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac39501-da86-4059-81a3-e5a40ebc0291 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:13.436118 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f620e47-9a1e-42cb-9b87-18ecf263119f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:13.436393 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f620e47-9a1e-42cb-9b87-18ecf263119f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:13.436626 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f620e47-9a1e-42cb-9b87-18ecf263119f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:13.436896 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f620e47-9a1e-42cb-9b87-18ecf263119f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:13.446174 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f620e47-9a1e-42cb-9b87-18ecf263119f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:13.447667 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f620e47-9a1e-42cb-9b87-18ecf263119f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:13.462643 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-7f620e47-9a1e-42cb-9b87-18ecf263119f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 16:54:13.462643 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:13.462643 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:13.462643 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:13.462643 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:13.462643 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:13.462643 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:13.462643 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:13.462643 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:13.462643 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:13.462643 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:13.462643 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:13.462643 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:13.462643 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:13.462643 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:13.462643 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:13.463805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:13.463805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:13.463805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:13.463805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:13.463805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:13.463805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:13.463805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:13.463805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:13.463805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:13.463805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:13.463805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:13.463805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:13.463805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:13.463805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:13.463805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:13.463805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:13.463805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:13.463805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:13.464925 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:13.464925 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:13.464925 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:13.464925 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:13.464925 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:13.464925 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:13.464925 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:13.464925 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:13.464925 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 16:54:13.464925 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:13.470103 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 296/595] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:54:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:13.471208 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f620e47-9a1e-42cb-9b87-18ecf263119f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:13.486677 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:13.549582 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac39501-da86-4059-81a3-e5a40ebc0291 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:13.578948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f620e47-9a1e-42cb-9b87-18ecf263119f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:13.587121 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ccd628-3bbb-44f9-bc09-f9bd25ea044e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:13.588008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ccd628-3bbb-44f9-bc09-f9bd25ea044e None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:13.588430 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ccd628-3bbb-44f9-bc09-f9bd25ea044e None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:13.588886 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ccd628-3bbb-44f9-bc09-f9bd25ea044e None admin] PATH_INFO: `/v3/projects` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:13.589906 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ccd628-3bbb-44f9-bc09-f9bd25ea044e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:13.590428 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ccd628-3bbb-44f9-bc09-f9bd25ea044e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:13.592112 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ccd628-3bbb-44f9-bc09-f9bd25ea044e None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:13.610731 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 300/596] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:13.612657 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ccd628-3bbb-44f9-bc09-f9bd25ea044e None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:13.614610 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270b275-b630-48b1-bf3f-193677c3e1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:13.615409 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270b275-b630-48b1-bf3f-193677c3e1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:13.615806 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270b275-b630-48b1-bf3f-193677c3e1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:13.616148 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3270b275-b630-48b1-bf3f-193677c3e1f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:13.617055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270b275-b630-48b1-bf3f-193677c3e1f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:13.617366 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270b275-b630-48b1-bf3f-193677c3e1f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:13.619026 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3270b275-b630-48b1-bf3f-193677c3e1f7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:13.632099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 297/597] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:13.633049 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270b275-b630-48b1-bf3f-193677c3e1f7 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:13.649216 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:13.706185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:13.727364 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ccd628-3bbb-44f9-bc09-f9bd25ea044e None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:13.738203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3270b275-b630-48b1-bf3f-193677c3e1f7 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:13.765284 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fd365-f9ba-4208-912a-26c8de7cd785 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:13.766681 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fd365-f9ba-4208-912a-26c8de7cd785 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:13.767255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fd365-f9ba-4208-912a-26c8de7cd785 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:13.767753 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408fd365-f9ba-4208-912a-26c8de7cd785 None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:13.768778 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408fd365-f9ba-4208-912a-26c8de7cd785 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:13.769294 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408fd365-f9ba-4208-912a-26c8de7cd785 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:13.771144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408fd365-f9ba-4208-912a-26c8de7cd785 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:13.779359 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9d751-0250-4128-924b-7d9c21ab7a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:13.780422 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9d751-0250-4128-924b-7d9c21ab7a03 None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:13.780703 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9d751-0250-4128-924b-7d9c21ab7a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:13.780976 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c9d751-0250-4128-924b-7d9c21ab7a03 None admin] PATH_INFO: `/v3/projects/0595667fb2e04a3ca4716f669d6698b0/users/d93036329cf64a8e91edaee5b1d47b9c/roles/5a7214dc069f45969e19c44122b378a4` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:13.795185 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 301/598] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 23 16:54:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:13.796368 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fd365-f9ba-4208-912a-26c8de7cd785 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:13.797610 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c9d751-0250-4128-924b-7d9c21ab7a03 None admin] RBAC: Authorizing `identity:create_grant(project_id=0595667fb2e04a3ca4716f669d6698b0, user_id=d93036329cf64a8e91edaee5b1d47b9c, role_id=5a7214dc069f45969e19c44122b378a4)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:13.799146 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c9d751-0250-4128-924b-7d9c21ab7a03 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:13.817802 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 298/599] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:54:13 2022] PUT /identity/v3/projects/0595667fb2e04a3ca4716f669d6698b0/users/d93036329cf64a8e91edaee5b1d47b9c/roles/5a7214dc069f45969e19c44122b378a4 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:54:13.818785 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9d751-0250-4128-924b-7d9c21ab7a03 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:13.934039 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:13.949386 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408fd365-f9ba-4208-912a-26c8de7cd785 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:13.956292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c9d751-0250-4128-924b-7d9c21ab7a03 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:13.982122 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:14.000093 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2bb50-658d-4287-a251-8a23ed4b797b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:14.001012 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2bb50-658d-4287-a251-8a23ed4b797b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:14.001295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2bb50-658d-4287-a251-8a23ed4b797b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:14.001573 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f2bb50-658d-4287-a251-8a23ed4b797b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:14.003600 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbaa13-db6e-4b7e-84b8-8db4c259af67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:14.005788 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbaa13-db6e-4b7e-84b8-8db4c259af67 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:14.006069 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbaa13-db6e-4b7e-84b8-8db4c259af67 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:14.006337 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fbaa13-db6e-4b7e-84b8-8db4c259af67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:14.013897 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f2bb50-658d-4287-a251-8a23ed4b797b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:14.015449 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f2bb50-658d-4287-a251-8a23ed4b797b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:14.022581 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbaa13-db6e-4b7e-84b8-8db4c259af67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:14.024561 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fbaa13-db6e-4b7e-84b8-8db4c259af67 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:14.031523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-f4f2bb50-658d-4287-a251-8a23ed4b797b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:14.031523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:14.031523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:14.031523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:14.031523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:14.031523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:14.031523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:14.031523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:14.031523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:14.031523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:14.031523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:14.031523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:14.031523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:14.031523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:14.031523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:14.032588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:14.032588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:14.032588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:14.032588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:14.032588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:14.032588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:14.032588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:14.032588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:14.032588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:14.032588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:14.032588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:14.032588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:14.032588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:14.032588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:14.032588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:14.032588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:14.032588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:14.033687 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:14.033687 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:14.033687 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:14.033687 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:14.033687 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:14.033687 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:14.033687 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:14.033687 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:14.035547 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 302/600] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:14.042127 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2bb50-658d-4287-a251-8a23ed4b797b None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:14.049196 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-81fbaa13-db6e-4b7e-84b8-8db4c259af67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:14.049196 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:14.049196 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:14.049196 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:14.049196 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:14.049196 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:14.049196 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:14.049196 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:14.049196 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:14.049196 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:14.049196 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:14.049196 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:14.049196 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:14.049196 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:14.049196 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:14.050273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:14.050273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:14.050273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:14.050273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:14.050273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:14.050273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:14.050273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:14.050273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:14.050273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:14.050273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:14.050273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:14.050273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:14.050273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:14.050273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:14.050273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:14.050273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:14.050273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:14.051391 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:14.051391 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:14.051391 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:14.051391 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:14.053396 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 299/601] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:14.060886 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbaa13-db6e-4b7e-84b8-8db4c259af67 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:14.190750 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fbaa13-db6e-4b7e-84b8-8db4c259af67 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:14.229634 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f2bb50-658d-4287-a251-8a23ed4b797b None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:14.234822 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a785a6e-0688-4ab4-907d-bda7f45d0941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:14.235801 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a785a6e-0688-4ab4-907d-bda7f45d0941 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:14.236267 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a785a6e-0688-4ab4-907d-bda7f45d0941 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:14.236685 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a785a6e-0688-4ab4-907d-bda7f45d0941 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:14.237574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a785a6e-0688-4ab4-907d-bda7f45d0941 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:14.238025 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a785a6e-0688-4ab4-907d-bda7f45d0941 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:14.239633 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a785a6e-0688-4ab4-907d-bda7f45d0941 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:14.255944 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 300/602] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:14.261640 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a785a6e-0688-4ab4-907d-bda7f45d0941 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:14.282826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd2255-bb24-4d7a-85dd-2457b3f2313a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:14.283664 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd2255-bb24-4d7a-85dd-2457b3f2313a None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:14.284016 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd2255-bb24-4d7a-85dd-2457b3f2313a None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:14.284423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadd2255-bb24-4d7a-85dd-2457b3f2313a None admin] PATH_INFO: `/v3/projects` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:14.285370 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd2255-bb24-4d7a-85dd-2457b3f2313a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:14.286938 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd2255-bb24-4d7a-85dd-2457b3f2313a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:14.295787 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadd2255-bb24-4d7a-85dd-2457b3f2313a None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:14.304818 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:14.334123 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 303/603] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:14.340055 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd2255-bb24-4d7a-85dd-2457b3f2313a None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:14.372801 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:14.388063 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a785a6e-0688-4ab4-907d-bda7f45d0941 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:14.428826 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00b113-50fa-4d1b-8ab9-580eeb40d043 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:14.431168 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00b113-50fa-4d1b-8ab9-580eeb40d043 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:14.432840 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00b113-50fa-4d1b-8ab9-580eeb40d043 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:14.433295 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c00b113-50fa-4d1b-8ab9-580eeb40d043 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:14.451717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00b113-50fa-4d1b-8ab9-580eeb40d043 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:14.453591 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c00b113-50fa-4d1b-8ab9-580eeb40d043 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:14.490609 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-0c00b113-50fa-4d1b-8ab9-580eeb40d043 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:14.490609 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:14.490609 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:14.490609 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:14.490609 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:14.490609 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:14.490609 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:14.490609 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:14.490609 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:14.490609 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:14.490609 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:14.490609 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:14.490609 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:14.490609 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:14.490609 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:14.491580 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:14.491580 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:14.491580 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:14.491580 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:14.491580 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:14.491580 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:14.491580 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:14.491580 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:14.491580 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:14.491580 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:14.491580 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:14.491580 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:14.491580 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:14.491580 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:14.491580 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:14.491580 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:14.491580 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:14.492465 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:14.492465 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:14.492465 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:14.492465 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:14.492465 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:14.492465 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:14.492465 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:14.492465 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:14.494902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 301/604] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:14.498220 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00b113-50fa-4d1b-8ab9-580eeb40d043 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:14.605070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadd2255-bb24-4d7a-85dd-2457b3f2313a None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:14.640729 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:14.653316 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c00b113-50fa-4d1b-8ab9-580eeb40d043 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:14.691858 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09a4ba1-5e17-40a0-9151-dcb3238e00e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:14.692772 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a4ba1-5e17-40a0-9151-dcb3238e00e0 None admin] REQUEST_METHOD: `PUT` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:14.693052 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a4ba1-5e17-40a0-9151-dcb3238e00e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:14.693221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea79205-7475-49a6-8d89-ed0a63d1ca24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:14.693415 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09a4ba1-5e17-40a0-9151-dcb3238e00e0 None admin] PATH_INFO: `/v3/projects/0595667fb2e04a3ca4716f669d6698b0/users/fd070e7581f04f4abcbb8b6bc231a8c1/roles/5a7214dc069f45969e19c44122b378a4` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:14.693721 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea79205-7475-49a6-8d89-ed0a63d1ca24 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:14.694008 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea79205-7475-49a6-8d89-ed0a63d1ca24 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:14.694280 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea79205-7475-49a6-8d89-ed0a63d1ca24 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:14.695101 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea79205-7475-49a6-8d89-ed0a63d1ca24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:14.695414 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea79205-7475-49a6-8d89-ed0a63d1ca24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:14.697921 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea79205-7475-49a6-8d89-ed0a63d1ca24 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:14.711290 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a4ba1-5e17-40a0-9151-dcb3238e00e0 None admin] RBAC: Authorizing `identity:create_grant(project_id=0595667fb2e04a3ca4716f669d6698b0, user_id=fd070e7581f04f4abcbb8b6bc231a8c1, role_id=5a7214dc069f45969e19c44122b378a4)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:14.712196 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 302/605] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:14.712654 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09a4ba1-5e17-40a0-9151-dcb3238e00e0 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:14.713554 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1281f56-bb2d-4b9f-9fdb-59b8821923ff None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:14.713843 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1281f56-bb2d-4b9f-9fdb-59b8821923ff None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:14.714121 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1281f56-bb2d-4b9f-9fdb-59b8821923ff None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:14.715804 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 303/606] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:14.717685 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1281f56-bb2d-4b9f-9fdb-59b8821923ff None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:14.735807 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 304/607] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:54:14 2022] PUT /identity/v3/projects/0595667fb2e04a3ca4716f669d6698b0/users/fd070e7581f04f4abcbb8b6bc231a8c1/roles/5a7214dc069f45969e19c44122b378a4 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:54:14.737567 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd51c0-9d5a-48ff-9f6e-6f2afc8d85d7 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:14.737854 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd51c0-9d5a-48ff-9f6e-6f2afc8d85d7 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:14.738126 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dd51c0-9d5a-48ff-9f6e-6f2afc8d85d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:14.759412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-e0dd51c0-9d5a-48ff-9f6e-6f2afc8d85d7 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:14.760824 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-e0dd51c0-9d5a-48ff-9f6e-6f2afc8d85d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:14.831035 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:14.848302 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 305/608] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:54:14.873511 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd51c0-9d5a-48ff-9f6e-6f2afc8d85d7 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:14.948573 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1281f56-bb2d-4b9f-9fdb-59b8821923ff None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:14.987913 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca3aa5-6e6f-4231-a5c3-5bf26f5c39ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:14.988895 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca3aa5-6e6f-4231-a5c3-5bf26f5c39ce None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:14.989367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca3aa5-6e6f-4231-a5c3-5bf26f5c39ce None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:14.989644 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ca3aa5-6e6f-4231-a5c3-5bf26f5c39ce None admin] PATH_INFO: `/v3/projects/0595667fb2e04a3ca4716f669d6698b0/users/22e45ab9d34c4fcbaf93be339257eca8/roles/5a7214dc069f45969e19c44122b378a4` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:14.994703 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dd51c0-9d5a-48ff-9f6e-6f2afc8d85d7 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:15.006033 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca3aa5-6e6f-4231-a5c3-5bf26f5c39ce None admin] RBAC: Authorizing `identity:create_grant(project_id=0595667fb2e04a3ca4716f669d6698b0, user_id=22e45ab9d34c4fcbaf93be339257eca8, role_id=5a7214dc069f45969e19c44122b378a4)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:15.007641 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ca3aa5-6e6f-4231-a5c3-5bf26f5c39ce None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:15.026506 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 304/609] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:54:14 2022] PUT /identity/v3/projects/0595667fb2e04a3ca4716f669d6698b0/users/22e45ab9d34c4fcbaf93be339257eca8/roles/5a7214dc069f45969e19c44122b378a4 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 16:54:15.032209 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d551437-5239-421e-87bd-2baf0ac1602b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:15.032945 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d551437-5239-421e-87bd-2baf0ac1602b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:15.033224 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d551437-5239-421e-87bd-2baf0ac1602b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:15.033496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d551437-5239-421e-87bd-2baf0ac1602b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:15.048171 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d551437-5239-421e-87bd-2baf0ac1602b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:15.056549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-3d551437-5239-421e-87bd-2baf0ac1602b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:15.056549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:15.056549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:15.056549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:15.056549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:15.056549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:15.056549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:15.056549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:15.056549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:15.056549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:15.056549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:15.056549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:15.056549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:15.056549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:15.056549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:15.056549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:15.057622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:15.057622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:15.057622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:15.057622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:15.057622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:15.057622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:15.057622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:15.057622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:15.057622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:15.057622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:15.057622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:15.057622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:15.057622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:15.057622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:15.057622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:15.057622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:15.058442 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:15.058442 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:15.058442 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:15.058442 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:15.058442 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:15.058442 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:15.058442 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:15.058442 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:15.058442 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:15.058442 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:15.058442 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:15.058442 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:15.058442 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 306/610] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 23 16:54:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:54:15.068175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca3aa5-6e6f-4231-a5c3-5bf26f5c39ce None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:15.071507 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:15.230027 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ca3aa5-6e6f-4231-a5c3-5bf26f5c39ce None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:15.280931 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239d14f7-0dd4-4a88-a0b2-870119739a8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:15.281588 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d14f7-0dd4-4a88-a0b2-870119739a8f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:15.281869 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d14f7-0dd4-4a88-a0b2-870119739a8f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:15.282141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239d14f7-0dd4-4a88-a0b2-870119739a8f None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:15.283115 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d14f7-0dd4-4a88-a0b2-870119739a8f None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:15.283435 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d14f7-0dd4-4a88-a0b2-870119739a8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:15.284825 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239d14f7-0dd4-4a88-a0b2-870119739a8f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:15.298762 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 305/611] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 23 16:54:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:15.304744 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d551437-5239-421e-87bd-2baf0ac1602b None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:15.380900 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:15.398213 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326064e-e082-42b2-92cd-14c0f0efe94d None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:15.398525 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326064e-e082-42b2-92cd-14c0f0efe94d None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:15.398762 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8326064e-e082-42b2-92cd-14c0f0efe94d None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:15.400179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 306/612] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:15.405630 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cadabd-d7cd-47b7-b028-4f3afd782709 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:15.405892 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cadabd-d7cd-47b7-b028-4f3afd782709 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:15.406160 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cadabd-d7cd-47b7-b028-4f3afd782709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:15.439157 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-07cadabd-d7cd-47b7-b028-4f3afd782709 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:15.440360 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-07cadabd-d7cd-47b7-b028-4f3afd782709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:15.446784 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d551437-5239-421e-87bd-2baf0ac1602b None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:15.537307 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:15.539490 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8542b-e286-4463-b3fb-47fc067d80fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:15.541715 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8542b-e286-4463-b3fb-47fc067d80fb None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:15.541715 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8542b-e286-4463-b3fb-47fc067d80fb None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:15.541715 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd8542b-e286-4463-b3fb-47fc067d80fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:15.548614 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 307/613] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:54:15.560875 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8542b-e286-4463-b3fb-47fc067d80fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:15.562452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd8542b-e286-4463-b3fb-47fc067d80fb None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:15.586322 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-5cd8542b-e286-4463-b3fb-47fc067d80fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:15.586322 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:15.586322 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:15.586322 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:15.586322 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:15.586322 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:15.586322 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:15.586322 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:15.586322 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:15.586322 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:15.586322 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:15.586322 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:15.586322 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:15.586322 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:15.586322 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:15.587345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:15.587345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:15.587345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:15.587345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:15.587345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:15.587345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:15.587345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:15.587345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:15.587345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:15.587345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:15.587345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:15.587345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:15.587345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:15.587345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:15.587345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:15.587345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:15.587345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:15.588265 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:15.588265 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:15.588265 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:15.588265 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:15.591999 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 307/614] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:15.597205 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cadabd-d7cd-47b7-b028-4f3afd782709 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:15.599545 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8542b-e286-4463-b3fb-47fc067d80fb None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:15.710858 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cadabd-d7cd-47b7-b028-4f3afd782709 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:15.749608 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b8b50-e8cc-404d-a3ba-e7dc829a6fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:15.750758 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b8b50-e8cc-404d-a3ba-e7dc829a6fc1 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:15.751277 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b8b50-e8cc-404d-a3ba-e7dc829a6fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:15.751821 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b8b50-e8cc-404d-a3ba-e7dc829a6fc1 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:15.752752 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b8b50-e8cc-404d-a3ba-e7dc829a6fc1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:15.753260 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b8b50-e8cc-404d-a3ba-e7dc829a6fc1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:15.755841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873b8b50-e8cc-404d-a3ba-e7dc829a6fc1 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:15.769008 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 308/615] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:15.774370 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b8b50-e8cc-404d-a3ba-e7dc829a6fc1 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:15.789452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd8542b-e286-4463-b3fb-47fc067d80fb None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:15.791355 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:15.854246 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c124c-6f58-4194-816c-18e016273eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:15.855910 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c124c-6f58-4194-816c-18e016273eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:15.856422 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c124c-6f58-4194-816c-18e016273eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:15.856891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006c124c-6f58-4194-816c-18e016273eb6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:15.880808 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:15.887794 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006c124c-6f58-4194-816c-18e016273eb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:15.899649 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-006c124c-6f58-4194-816c-18e016273eb6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:15.899649 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:15.899649 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:15.899649 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:15.899649 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:15.899649 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:15.899649 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:15.899649 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:15.899649 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:15.899649 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:15.899649 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:15.899649 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:15.899649 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:15.899649 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:15.899649 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:15.899649 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:15.900618 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:15.900618 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:15.900618 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:15.900618 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:15.900618 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:15.900618 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:15.900618 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:15.900618 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:15.900618 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:15.900618 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:15.900618 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:15.900618 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:15.900618 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:15.900618 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:15.900618 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:15.900618 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:15.902055 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:15.902055 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:15.902055 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:15.902055 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:15.902055 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:15.902055 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:15.902055 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:15.902055 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:15.902055 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:15.902055 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:15.902055 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:15.902055 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:15.902055 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 308/616] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 23 16:54:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:54:15.911919 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c124c-6f58-4194-816c-18e016273eb6 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:15.935679 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b8b50-e8cc-404d-a3ba-e7dc829a6fc1 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:15.973960 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d09b8-d1a6-4831-9e74-105af13bbd3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:15.975237 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d09b8-d1a6-4831-9e74-105af13bbd3b None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:15.975717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d09b8-d1a6-4831-9e74-105af13bbd3b None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:15.976199 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d09b8-d1a6-4831-9e74-105af13bbd3b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:15.986156 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d09b8-d1a6-4831-9e74-105af13bbd3b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:15.988138 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-014d09b8-d1a6-4831-9e74-105af13bbd3b None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:16.005251 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-014d09b8-d1a6-4831-9e74-105af13bbd3b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 16:54:16.005251 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:16.005251 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:16.005251 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:16.005251 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:16.005251 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:16.005251 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:16.005251 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:16.005251 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:16.005251 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:16.005251 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:16.005251 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:16.005251 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:16.005251 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:16.005251 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:16.005251 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:16.006266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:16.006266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:16.006266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:16.006266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:16.006266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:16.006266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:16.006266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:16.006266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:16.006266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:16.006266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:16.006266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:16.006266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:16.006266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:16.006266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:16.006266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:16.006266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:16.006266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:16.006266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:16.007236 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:16.007236 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:16.007236 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:16.007236 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:16.007236 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:16.007236 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:16.007236 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:16.007236 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:16.007236 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 16:54:16.007236 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:16.010350 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 309/617] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:54:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:16.020294 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d09b8-d1a6-4831-9e74-105af13bbd3b None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:16.105450 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006c124c-6f58-4194-816c-18e016273eb6 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:16.148613 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d7695-6b1d-47af-8786-3fc1fa8cad4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:16.149289 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d7695-6b1d-47af-8786-3fc1fa8cad4a None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:16.149566 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d7695-6b1d-47af-8786-3fc1fa8cad4a None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:16.149836 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07d7695-6b1d-47af-8786-3fc1fa8cad4a None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:16.150585 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d7695-6b1d-47af-8786-3fc1fa8cad4a None admin] RBAC: Adding query filter params (name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:16.150902 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d7695-6b1d-47af-8786-3fc1fa8cad4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:16.156184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07d7695-6b1d-47af-8786-3fc1fa8cad4a None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:16.173779 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 309/618] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 23 16:54:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:16.179219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d7695-6b1d-47af-8786-3fc1fa8cad4a None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:16.233170 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:16.239224 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d09b8-d1a6-4831-9e74-105af13bbd3b None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:16.255883 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:16.296664 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6c88a-1939-4591-b29d-cf9d7efac16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:16.297340 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6c88a-1939-4591-b29d-cf9d7efac16c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:16.297617 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6c88a-1939-4591-b29d-cf9d7efac16c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:16.297888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6c88a-1939-4591-b29d-cf9d7efac16c None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:16.298718 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6c88a-1939-4591-b29d-cf9d7efac16c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:16.299050 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6c88a-1939-4591-b29d-cf9d7efac16c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:16.300486 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6c88a-1939-4591-b29d-cf9d7efac16c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:16.323187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 310/619] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 23 16:54:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:16.324508 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6c88a-1939-4591-b29d-cf9d7efac16c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:16.374132 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07d7695-6b1d-47af-8786-3fc1fa8cad4a None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:16.417255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec99a85-6694-40f4-90db-e92855466104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:16.418065 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec99a85-6694-40f4-90db-e92855466104 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:16.418347 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec99a85-6694-40f4-90db-e92855466104 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:16.418620 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec99a85-6694-40f4-90db-e92855466104 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:16.441394 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec99a85-6694-40f4-90db-e92855466104 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:16.443094 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec99a85-6694-40f4-90db-e92855466104 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:16.478973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-dec99a85-6694-40f4-90db-e92855466104 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:16.478973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:16.478973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:16.478973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:16.478973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:16.478973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:16.478973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:16.478973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:16.478973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:16.478973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:16.478973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:16.478973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:16.478973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:16.478973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:16.478973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:16.479947 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:16.479947 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:16.479947 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:16.479947 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:16.479947 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:16.479947 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:16.479947 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:16.479947 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:16.479947 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:16.479947 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:16.479947 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:16.479947 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:16.479947 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:16.479947 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:16.479947 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:16.479947 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:16.479947 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:16.481851 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:16.481851 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:16.481851 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:16.481851 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:16.481851 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 310/620] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 302 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:16.491625 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec99a85-6694-40f4-90db-e92855466104 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:16.520814 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6c88a-1939-4591-b29d-cf9d7efac16c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:16.562437 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291b183-0771-4a2c-afc5-17e8e422116b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:16.563282 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291b183-0771-4a2c-afc5-17e8e422116b None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:16.563565 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291b183-0771-4a2c-afc5-17e8e422116b None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:16.563840 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3291b183-0771-4a2c-afc5-17e8e422116b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:16.575915 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291b183-0771-4a2c-afc5-17e8e422116b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:16.577403 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3291b183-0771-4a2c-afc5-17e8e422116b None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:16.597108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-3291b183-0771-4a2c-afc5-17e8e422116b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:16.597108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:16.597108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:16.597108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:16.597108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:16.597108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:16.597108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:16.597108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:16.597108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:16.597108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:16.597108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:16.597108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:16.597108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:16.597108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:16.597108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:16.598933 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:16.598933 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:16.598933 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:16.598933 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:16.598933 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:16.598933 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:16.598933 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:16.598933 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:16.598933 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:16.598933 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:16.598933 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:16.598933 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:16.598933 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:16.598933 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:16.598933 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:16.598933 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:16.598933 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:16.600225 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:16.600225 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:16.600225 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:16.600225 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:16.600225 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 311/621] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:54:16.604818 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291b183-0771-4a2c-afc5-17e8e422116b None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:16.616343 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:16.656615 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:16.657036 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec99a85-6694-40f4-90db-e92855466104 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:16.732453 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8203f48-5391-460f-b3e2-87032a5f3fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:16.733644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8203f48-5391-460f-b3e2-87032a5f3fb1 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:16.735441 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8203f48-5391-460f-b3e2-87032a5f3fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:16.736899 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8203f48-5391-460f-b3e2-87032a5f3fb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:16.737842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8203f48-5391-460f-b3e2-87032a5f3fb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:16.738976 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8203f48-5391-460f-b3e2-87032a5f3fb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:16.742449 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8203f48-5391-460f-b3e2-87032a5f3fb1 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:16.757988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3291b183-0771-4a2c-afc5-17e8e422116b None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:16.772524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 311/622] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:16.777634 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8203f48-5391-460f-b3e2-87032a5f3fb1 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:16.821797 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d1b540-f27e-449f-ae68-186a3c5f797b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:16.823045 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1b540-f27e-449f-ae68-186a3c5f797b None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:16.823330 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1b540-f27e-449f-ae68-186a3c5f797b None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:16.823610 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d1b540-f27e-449f-ae68-186a3c5f797b None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:16.824339 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1b540-f27e-449f-ae68-186a3c5f797b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:16.824645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1b540-f27e-449f-ae68-186a3c5f797b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:16.826084 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d1b540-f27e-449f-ae68-186a3c5f797b None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:16.841484 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 312/623] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:16.842473 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999808f1-27d4-4051-905d-46363b5de081 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:16.842764 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999808f1-27d4-4051-905d-46363b5de081 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:16.843067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999808f1-27d4-4051-905d-46363b5de081 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:16.848649 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 313/624] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:16.848649 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999808f1-27d4-4051-905d-46363b5de081 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:16.932908 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8203f48-5391-460f-b3e2-87032a5f3fb1 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:16.957875 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:16.977375 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043c1f24-6e0a-4197-a96d-f9ae02f668d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:16.979207 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c1f24-6e0a-4197-a96d-f9ae02f668d6 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:16.979696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c1f24-6e0a-4197-a96d-f9ae02f668d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:16.980200 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043c1f24-6e0a-4197-a96d-f9ae02f668d6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:16.981658 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999808f1-27d4-4051-905d-46363b5de081 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:16.984494 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:16.991289 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c1f24-6e0a-4197-a96d-f9ae02f668d6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:16.993520 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043c1f24-6e0a-4197-a96d-f9ae02f668d6 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:17.012934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-043c1f24-6e0a-4197-a96d-f9ae02f668d6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 16:54:17.012934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:17.012934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:17.012934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:17.012934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:17.012934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:17.012934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:17.012934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:17.012934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:17.012934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:17.012934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:17.012934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:17.012934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:17.012934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:17.012934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:17.012934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:17.013941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:17.013941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:17.013941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:17.013941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:17.013941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:17.013941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:17.013941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:17.013941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:17.013941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:17.013941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:17.013941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:17.013941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:17.013941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:17.013941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:17.013941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:17.013941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:17.013941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:17.013941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:17.014929 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:17.014929 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:17.014929 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:17.014929 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:17.014929 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:17.014929 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:17.014929 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:17.014929 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:17.014929 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 16:54:17.014929 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:17.021221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 312/625] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 23 16:54:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:54:17.021221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d5893-19ea-4e34-ab9a-e053bd81c4eb None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:17.021221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d5893-19ea-4e34-ab9a-e053bd81c4eb None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:17.021221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026d5893-19ea-4e34-ab9a-e053bd81c4eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:17.030091 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e2f8a-72d9-42e7-9d75-d440923e5df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:17.030893 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e2f8a-72d9-42e7-9d75-d440923e5df8 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:17.031205 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e2f8a-72d9-42e7-9d75-d440923e5df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:17.031478 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025e2f8a-72d9-42e7-9d75-d440923e5df8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:17.040297 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025e2f8a-72d9-42e7-9d75-d440923e5df8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:17.041791 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025e2f8a-72d9-42e7-9d75-d440923e5df8 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:17.043267 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-026d5893-19ea-4e34-ab9a-e053bd81c4eb None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:17.045205 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-026d5893-19ea-4e34-ab9a-e053bd81c4eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:17.060291 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-025e2f8a-72d9-42e7-9d75-d440923e5df8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:17.060291 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:17.060291 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:17.060291 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:17.060291 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:17.060291 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:17.060291 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:17.060291 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:17.060291 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:17.060291 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:17.060291 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:17.060291 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:17.060291 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:17.060291 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:17.060291 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:17.061292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:17.061292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:17.061292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:17.061292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:17.061292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:17.061292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:17.061292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:17.061292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:17.061292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:17.061292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:17.061292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:17.061292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:17.061292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:17.061292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:17.061292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:17.061292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:17.061292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:17.062173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:17.062173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:17.062173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:17.062173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:17.062173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:17.062173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:17.062173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:17.062173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:17.062173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 314/626] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:17.063203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e2f8a-72d9-42e7-9d75-d440923e5df8 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:17.139219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 313/627] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:54:17.142591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d5893-19ea-4e34-ab9a-e053bd81c4eb None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:17.211767 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025e2f8a-72d9-42e7-9d75-d440923e5df8 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:17.248885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8590f-8841-463b-9c00-447138b36ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:17.249567 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8590f-8841-463b-9c00-447138b36ede None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:17.249846 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8590f-8841-463b-9c00-447138b36ede None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:17.250120 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a8590f-8841-463b-9c00-447138b36ede None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:17.250984 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8590f-8841-463b-9c00-447138b36ede None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:17.251300 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8590f-8841-463b-9c00-447138b36ede None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:17.252750 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46a8590f-8841-463b-9c00-447138b36ede None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:17.259834 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026d5893-19ea-4e34-ab9a-e053bd81c4eb None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:17.273976 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 315/628] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 23 16:54:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:17.275008 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8590f-8841-463b-9c00-447138b36ede None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:17.279308 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:17.299289 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:17.308574 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed801de5-8f92-424e-896e-b6ef25d7b562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:17.309244 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed801de5-8f92-424e-896e-b6ef25d7b562 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:17.309533 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed801de5-8f92-424e-896e-b6ef25d7b562 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:17.309809 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed801de5-8f92-424e-896e-b6ef25d7b562 None admin] PATH_INFO: `/v3/projects` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:17.310599 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed801de5-8f92-424e-896e-b6ef25d7b562 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:17.310943 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed801de5-8f92-424e-896e-b6ef25d7b562 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:17.312417 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed801de5-8f92-424e-896e-b6ef25d7b562 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:17.327049 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 314/629] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:17.327049 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed801de5-8f92-424e-896e-b6ef25d7b562 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:17.384336 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a8590f-8841-463b-9c00-447138b36ede None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:17.421139 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783316cd-6721-4e29-bb3a-25409b5f7b5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:17.421938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783316cd-6721-4e29-bb3a-25409b5f7b5f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:17.422214 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783316cd-6721-4e29-bb3a-25409b5f7b5f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:17.422870 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783316cd-6721-4e29-bb3a-25409b5f7b5f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:17.433502 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783316cd-6721-4e29-bb3a-25409b5f7b5f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:17.435111 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed801de5-8f92-424e-896e-b6ef25d7b562 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:17.441489 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-783316cd-6721-4e29-bb3a-25409b5f7b5f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:17.441489 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:17.441489 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:17.441489 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:17.441489 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:17.441489 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:17.441489 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:17.441489 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:17.441489 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:17.441489 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:17.441489 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:17.441489 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:17.441489 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:17.441489 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:17.441489 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:17.441489 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:17.442612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:17.442612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:17.442612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:17.442612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:17.442612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:17.442612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:17.442612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:17.442612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:17.442612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:17.442612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:17.442612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:17.442612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:17.442612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:17.442612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:17.442612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:17.442612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:17.449005 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:17.449005 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:17.449005 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:17.449005 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:17.449005 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:17.449005 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:17.449005 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:17.449005 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:17.449005 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:17.449005 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:17.449005 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:17.449005 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:17.449005 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 316/630] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 23 16:54:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:17.449005 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9978103-88a9-40ed-8858-259ec77f7005 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:17.449777 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9978103-88a9-40ed-8858-259ec77f7005 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:17.449777 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9978103-88a9-40ed-8858-259ec77f7005 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:17.449777 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 317/631] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:17.449777 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9978103-88a9-40ed-8858-259ec77f7005 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:17.489584 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895f84cf-6b3f-411e-abe8-340a5ca18486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:17.490251 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f84cf-6b3f-411e-abe8-340a5ca18486 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:17.490535 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f84cf-6b3f-411e-abe8-340a5ca18486 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:17.490811 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895f84cf-6b3f-411e-abe8-340a5ca18486 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:17.516792 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:17.553988 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:17.558222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9978103-88a9-40ed-8858-259ec77f7005 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:17.569680 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f84cf-6b3f-411e-abe8-340a5ca18486 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:17.571466 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895f84cf-6b3f-411e-abe8-340a5ca18486 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:17.595613 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f99198-4470-40d7-8657-b106419581b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:17.596523 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f99198-4470-40d7-8657-b106419581b7 None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:17.596805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f99198-4470-40d7-8657-b106419581b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:17.597079 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f99198-4470-40d7-8657-b106419581b7 None admin] PATH_INFO: `/v3/projects/0595667fb2e04a3ca4716f669d6698b0/users/6546217667054abb8eb07076995c1b04/roles/5a7214dc069f45969e19c44122b378a4` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:17.614314 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f99198-4470-40d7-8657-b106419581b7 None admin] RBAC: Authorizing `identity:create_grant(project_id=0595667fb2e04a3ca4716f669d6698b0, user_id=6546217667054abb8eb07076995c1b04, role_id=5a7214dc069f45969e19c44122b378a4)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:17.615806 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f99198-4470-40d7-8657-b106419581b7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:17.630795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 318/632] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:54:17 2022] PUT /identity/v3/projects/0595667fb2e04a3ca4716f669d6698b0/users/6546217667054abb8eb07076995c1b04/roles/5a7214dc069f45969e19c44122b378a4 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:54:17.632407 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f99198-4470-40d7-8657-b106419581b7 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:17.680900 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 315/633] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:54:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:54:17.682262 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c049a-b6f5-4e28-b3cf-87a167397055 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:17.682548 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c049a-b6f5-4e28-b3cf-87a167397055 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:17.682818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39c049a-b6f5-4e28-b3cf-87a167397055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:17.701948 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-b39c049a-b6f5-4e28-b3cf-87a167397055 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:17.703272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-b39c049a-b6f5-4e28-b3cf-87a167397055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:17.757196 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f99198-4470-40d7-8657-b106419581b7 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:17.773146 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:17.780152 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 316/634] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:54:17.789124 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db597d-8f39-497f-9008-0c95ba1e1035 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:17.790019 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db597d-8f39-497f-9008-0c95ba1e1035 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:17.790548 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5db597d-8f39-497f-9008-0c95ba1e1035 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:17.797209 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 317/635] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:17 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:17.797209 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db597d-8f39-497f-9008-0c95ba1e1035 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:17.807516 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abb608c1-539b-4c26-bbff-4ae636f8fe3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:17.808234 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb608c1-539b-4c26-bbff-4ae636f8fe3a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:17.808567 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb608c1-539b-4c26-bbff-4ae636f8fe3a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:17.808842 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abb608c1-539b-4c26-bbff-4ae636f8fe3a None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:17.809600 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb608c1-539b-4c26-bbff-4ae636f8fe3a None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:17.809913 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb608c1-539b-4c26-bbff-4ae636f8fe3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:17.811402 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abb608c1-539b-4c26-bbff-4ae636f8fe3a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:17.820718 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:17.823966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 319/636] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 23 16:54:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:17.824807 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137335c8-df5e-474c-9e3c-56fc5d1f038b None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:17.824807 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137335c8-df5e-474c-9e3c-56fc5d1f038b None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:17.825132 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137335c8-df5e-474c-9e3c-56fc5d1f038b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:17.844252 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-137335c8-df5e-474c-9e3c-56fc5d1f038b None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:17.845434 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-137335c8-df5e-474c-9e3c-56fc5d1f038b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:17.914751 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5db597d-8f39-497f-9008-0c95ba1e1035 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:17.922931 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 320/637] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:54:17.922931 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137335c8-df5e-474c-9e3c-56fc5d1f038b None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:17.964759 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d94a8-464f-4f4d-81bd-3a91cfbfb355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:17.965672 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d94a8-464f-4f4d-81bd-3a91cfbfb355 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:17.966020 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d94a8-464f-4f4d-81bd-3a91cfbfb355 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:17.967125 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231d94a8-464f-4f4d-81bd-3a91cfbfb355 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:17.977899 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d94a8-464f-4f4d-81bd-3a91cfbfb355 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:17.979431 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231d94a8-464f-4f4d-81bd-3a91cfbfb355 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:17.997603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-231d94a8-464f-4f4d-81bd-3a91cfbfb355 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:17.997603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:17.997603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:17.997603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:17.997603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:17.997603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:17.997603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:17.997603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:17.997603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:17.997603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:17.997603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:17.997603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:17.997603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:17.997603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:17.997603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:17.998572 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:17.998572 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:17.998572 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:17.998572 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:17.998572 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:17.998572 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:17.998572 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:17.998572 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:17.998572 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:17.998572 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:17.998572 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:17.998572 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:17.998572 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:17.998572 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:17.998572 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:17.998572 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:17.998572 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:17.999777 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:17.999777 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:17.999777 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:17.999777 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:17.999777 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 318/638] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:17.999777 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d94a8-464f-4f4d-81bd-3a91cfbfb355 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:18.045900 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137335c8-df5e-474c-9e3c-56fc5d1f038b None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:18.049097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:18.067758 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:18.085742 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686ce0e-7ed8-412f-a91d-fa516cb1d49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:18.086554 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686ce0e-7ed8-412f-a91d-fa516cb1d49b None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:18.086838 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686ce0e-7ed8-412f-a91d-fa516cb1d49b None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:18.087160 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5686ce0e-7ed8-412f-a91d-fa516cb1d49b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:18.098782 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5686ce0e-7ed8-412f-a91d-fa516cb1d49b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:18.103099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231d94a8-464f-4f4d-81bd-3a91cfbfb355 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:18.106448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-5686ce0e-7ed8-412f-a91d-fa516cb1d49b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:18.106448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:18.106448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:18.106448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:18.106448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:18.106448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:18.106448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:18.106448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:18.106448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:18.106448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:18.106448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:18.106448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:18.106448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:18.106448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:18.106448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:18.106448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:18.107441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:18.107441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:18.107441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:18.107441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:18.107441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:18.107441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:18.107441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:18.107441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:18.107441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:18.107441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:18.107441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:18.107441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:18.107441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:18.107441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:18.107441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:18.107441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:18.114574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:18.114574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:18.114574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:18.114574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:18.114574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:18.114574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:18.114574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:18.114574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:18.114574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:18.114574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:18.114574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:18.114574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:18.114574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 321/639] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 23 16:54:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:18.114574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686ce0e-7ed8-412f-a91d-fa516cb1d49b None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:18.145917 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82466a8-c879-4c71-b23d-0b2f60428d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:18.146777 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82466a8-c879-4c71-b23d-0b2f60428d45 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:18.147099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82466a8-c879-4c71-b23d-0b2f60428d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:18.147371 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82466a8-c879-4c71-b23d-0b2f60428d45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:18.161866 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82466a8-c879-4c71-b23d-0b2f60428d45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:18.163434 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82466a8-c879-4c71-b23d-0b2f60428d45 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:18.181281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-f82466a8-c879-4c71-b23d-0b2f60428d45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:18.181281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:18.181281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:18.181281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:18.181281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:18.181281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:18.181281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:18.181281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:18.181281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:18.181281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:18.181281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:18.181281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:18.181281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:18.181281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:18.181281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:18.182211 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:18.182211 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:18.182211 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:18.182211 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:18.182211 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:18.182211 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:18.182211 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:18.182211 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:18.182211 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:18.182211 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:18.182211 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:18.182211 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:18.182211 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:18.182211 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:18.182211 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:18.182211 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:18.182211 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:18.183241 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:18.183241 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:18.183241 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:18.183241 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:18.183241 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 319/640] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:18.183548 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82466a8-c879-4c71-b23d-0b2f60428d45 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:18.228448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5686ce0e-7ed8-412f-a91d-fa516cb1d49b None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:18.263688 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098924ec-c069-4106-acce-1e0e1a099f41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:18.264532 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098924ec-c069-4106-acce-1e0e1a099f41 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:18.264810 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098924ec-c069-4106-acce-1e0e1a099f41 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:18.265087 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098924ec-c069-4106-acce-1e0e1a099f41 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:18.275318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098924ec-c069-4106-acce-1e0e1a099f41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:18.283013 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-098924ec-c069-4106-acce-1e0e1a099f41 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:18.283013 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:18.283013 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:18.283013 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:18.283013 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:18.283013 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:18.283013 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:18.283013 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:18.283013 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:18.283013 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:18.283013 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:18.283013 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:18.283013 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:18.283013 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:18.283013 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:18.283013 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:18.284038 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:18.284038 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:18.284038 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:18.284038 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:18.284038 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:18.284038 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:18.284038 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:18.284038 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:18.284038 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:18.284038 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:18.284038 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:18.284038 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:18.284038 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:18.284038 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:18.284038 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:18.284038 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:18.284924 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:18.284924 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:18.284924 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:18.284924 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:18.284924 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:18.284924 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:18.284924 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:18.284924 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:18.284924 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:18.284924 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:18.284924 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:18.284924 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:18.284924 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 322/641] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 23 16:54:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:18.285646 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098924ec-c069-4106-acce-1e0e1a099f41 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:18.297734 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82466a8-c879-4c71-b23d-0b2f60428d45 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:18.316852 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:18.318173 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:18.334444 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7a044-8fe8-4987-aa76-12fd5d68dd74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:18.335150 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7a044-8fe8-4987-aa76-12fd5d68dd74 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:18.335432 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7a044-8fe8-4987-aa76-12fd5d68dd74 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:18.335703 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2be7a044-8fe8-4987-aa76-12fd5d68dd74 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:18.336661 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be7a044-8fe8-4987-aa76-12fd5d68dd74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:18.337114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be7a044-8fe8-4987-aa76-12fd5d68dd74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:18.338331 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2be7a044-8fe8-4987-aa76-12fd5d68dd74 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:18.352014 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 320/642] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:18.352775 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7a044-8fe8-4987-aa76-12fd5d68dd74 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:18.409407 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098924ec-c069-4106-acce-1e0e1a099f41 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:18.448358 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2140157-462f-4dd0-aab4-84aef5c345f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:18.449037 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2140157-462f-4dd0-aab4-84aef5c345f0 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:18.449321 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2140157-462f-4dd0-aab4-84aef5c345f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:18.449599 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2140157-462f-4dd0-aab4-84aef5c345f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:18.450467 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2140157-462f-4dd0-aab4-84aef5c345f0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:18.450778 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2140157-462f-4dd0-aab4-84aef5c345f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:18.452233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2140157-462f-4dd0-aab4-84aef5c345f0 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:18.464876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 323/643] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 23 16:54:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:18.464876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2140157-462f-4dd0-aab4-84aef5c345f0 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:18.475957 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2be7a044-8fe8-4987-aa76-12fd5d68dd74 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:18.513138 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82704791-a1de-42c5-894e-2ad86e14dc1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:18.513917 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82704791-a1de-42c5-894e-2ad86e14dc1d None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:18.514203 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82704791-a1de-42c5-894e-2ad86e14dc1d None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:18.514481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82704791-a1de-42c5-894e-2ad86e14dc1d None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:18.515256 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82704791-a1de-42c5-894e-2ad86e14dc1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:18.515563 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82704791-a1de-42c5-894e-2ad86e14dc1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:18.516953 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82704791-a1de-42c5-894e-2ad86e14dc1d None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:18.528725 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 321/644] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:18.529337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82704791-a1de-42c5-894e-2ad86e14dc1d None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:18.553481 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:18.569886 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2140157-462f-4dd0-aab4-84aef5c345f0 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:18.572559 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:18.606613 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83497547-a461-4a1f-9d4e-4065c0644de1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:18.607322 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83497547-a461-4a1f-9d4e-4065c0644de1 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:18.607601 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83497547-a461-4a1f-9d4e-4065c0644de1 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:18.607874 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83497547-a461-4a1f-9d4e-4065c0644de1 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:18.608615 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83497547-a461-4a1f-9d4e-4065c0644de1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:18.608924 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83497547-a461-4a1f-9d4e-4065c0644de1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:18.610711 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83497547-a461-4a1f-9d4e-4065c0644de1 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:18.619792 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 324/645] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 23 16:54:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:18.620404 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83497547-a461-4a1f-9d4e-4065c0644de1 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:18.639616 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82704791-a1de-42c5-894e-2ad86e14dc1d None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:18.676497 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3bfd6-12ef-4d84-8683-825166d47778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:18.677401 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3bfd6-12ef-4d84-8683-825166d47778 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:18.677690 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3bfd6-12ef-4d84-8683-825166d47778 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:18.678078 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a3bfd6-12ef-4d84-8683-825166d47778 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:18.685343 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a3bfd6-12ef-4d84-8683-825166d47778 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:18.686798 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a3bfd6-12ef-4d84-8683-825166d47778 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:18.701944 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-b3a3bfd6-12ef-4d84-8683-825166d47778 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:18.701944 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:18.701944 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:18.701944 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:18.701944 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:18.701944 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:18.701944 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:18.701944 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:18.701944 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:18.701944 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:18.701944 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:18.701944 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:18.701944 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:18.701944 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:18.701944 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:18.702891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:18.702891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:18.702891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:18.702891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:18.702891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:18.702891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:18.702891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:18.702891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:18.702891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:18.702891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:18.702891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:18.702891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:18.702891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:18.702891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:18.702891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:18.702891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:18.702891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:18.704007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:18.704007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:18.704007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:18.704007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:18.704007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:18.704007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:18.704007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:18.704007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:18.704007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 322/646] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:18.704007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3bfd6-12ef-4d84-8683-825166d47778 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:18.725526 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83497547-a461-4a1f-9d4e-4065c0644de1 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:18.762894 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607a180-1916-4697-b751-42dcda5740d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:18.763748 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607a180-1916-4697-b751-42dcda5740d0 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:18.764029 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607a180-1916-4697-b751-42dcda5740d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:18.764301 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2607a180-1916-4697-b751-42dcda5740d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:18.775565 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2607a180-1916-4697-b751-42dcda5740d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:18.777018 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2607a180-1916-4697-b751-42dcda5740d0 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:18.785171 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:18.796182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-2607a180-1916-4697-b751-42dcda5740d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:18.796182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:18.796182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:18.796182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:18.796182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:18.796182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:18.796182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:18.796182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:18.796182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:18.796182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:18.796182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:18.796182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:18.796182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:18.796182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:18.796182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:18.797255 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:18.797255 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:18.797255 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:18.797255 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:18.797255 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:18.797255 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:18.797255 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:18.797255 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:18.797255 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:18.797255 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:18.797255 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:18.797255 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:18.797255 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:18.797255 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:18.797255 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:18.797255 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:18.797255 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:18.798207 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:18.798207 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:18.798207 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:18.798207 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:18.798207 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 325/647] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:18.798724 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607a180-1916-4697-b751-42dcda5740d0 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:18.816316 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:18.819225 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a3bfd6-12ef-4d84-8683-825166d47778 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:18.854989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74946fe4-1674-4aa9-8943-ac4708ff997e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:18.856035 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74946fe4-1674-4aa9-8943-ac4708ff997e None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:18.856487 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74946fe4-1674-4aa9-8943-ac4708ff997e None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:18.856904 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74946fe4-1674-4aa9-8943-ac4708ff997e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:18.865453 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74946fe4-1674-4aa9-8943-ac4708ff997e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:18.867138 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74946fe4-1674-4aa9-8943-ac4708ff997e None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:18.882641 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-74946fe4-1674-4aa9-8943-ac4708ff997e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 23 16:54:18.882641 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:18.882641 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:18.882641 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:18.882641 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:18.882641 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:18.882641 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:18.882641 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:18.882641 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:18.882641 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:18.882641 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:18.882641 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:18.882641 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:18.882641 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:18.882641 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:18.882641 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:18.884069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:18.884069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:18.884069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:18.884069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:18.884069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:18.884069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:18.884069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:18.884069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:18.884069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:18.884069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:18.884069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:18.884069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:18.884069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:18.884069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:18.884069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:18.884069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:18.884069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:18.884069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:18.885166 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:18.885166 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:18.885166 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:18.885166 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:18.885166 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:18.885166 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:18.885166 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:18.885166 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:18.885166 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 23 16:54:18.885166 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:18.888657 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 323/648] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:54:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:18.889632 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74946fe4-1674-4aa9-8943-ac4708ff997e None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:18.911269 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2607a180-1916-4697-b751-42dcda5740d0 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:18.945226 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c423e-8d2c-40ff-a7fb-a89b71f22ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:18.946021 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c423e-8d2c-40ff-a7fb-a89b71f22ef5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:18.946298 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c423e-8d2c-40ff-a7fb-a89b71f22ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:18.946564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6c423e-8d2c-40ff-a7fb-a89b71f22ef5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:18.957353 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6c423e-8d2c-40ff-a7fb-a89b71f22ef5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:18.958538 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6c423e-8d2c-40ff-a7fb-a89b71f22ef5 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:18.975365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-8e6c423e-8d2c-40ff-a7fb-a89b71f22ef5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:18.975365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:18.975365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:18.975365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:18.975365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:18.975365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:18.975365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:18.975365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:18.975365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:18.975365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:18.975365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:18.975365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:18.975365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:18.975365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:18.975365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:18.976261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:18.976261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:18.976261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:18.976261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:18.976261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:18.976261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:18.976261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:18.976261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:18.976261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:18.976261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:18.976261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:18.976261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:18.976261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:18.976261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:18.976261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:18.976261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:18.976261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:18.977134 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:18.977134 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:18.977134 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:18.977134 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:18.977134 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 326/649] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:18.977427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c423e-8d2c-40ff-a7fb-a89b71f22ef5 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:18.998872 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74946fe4-1674-4aa9-8943-ac4708ff997e None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:19.033110 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c36f3-3989-4212-8a87-76360e813b2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:19.033780 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c36f3-3989-4212-8a87-76360e813b2b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:19.034060 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c36f3-3989-4212-8a87-76360e813b2b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:19.034333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c36f3-3989-4212-8a87-76360e813b2b None admin] PATH_INFO: `/v3/projects` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:19.035488 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c36f3-3989-4212-8a87-76360e813b2b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:19.035488 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c36f3-3989-4212-8a87-76360e813b2b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:19.036923 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c36f3-3989-4212-8a87-76360e813b2b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:19.043314 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:19.048940 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 324/650] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:19.054596 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c36f3-3989-4212-8a87-76360e813b2b None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:19.079420 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:19.098428 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6c423e-8d2c-40ff-a7fb-a89b71f22ef5 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:19.134595 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47d822-678d-4770-a65b-78c900d8f641 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:19.135300 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47d822-678d-4770-a65b-78c900d8f641 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:19.135582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47d822-678d-4770-a65b-78c900d8f641 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:19.135856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f47d822-678d-4770-a65b-78c900d8f641 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:19.136581 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47d822-678d-4770-a65b-78c900d8f641 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:19.136883 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47d822-678d-4770-a65b-78c900d8f641 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:19.138300 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f47d822-678d-4770-a65b-78c900d8f641 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:19.155534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 327/651] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:19.155534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47d822-678d-4770-a65b-78c900d8f641 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:19.169076 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c36f3-3989-4212-8a87-76360e813b2b None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:19.205213 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc137bc-d5e0-494b-8aa9-7c4faf30c6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:19.206151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc137bc-d5e0-494b-8aa9-7c4faf30c6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:19.206436 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc137bc-d5e0-494b-8aa9-7c4faf30c6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:19.206723 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc137bc-d5e0-494b-8aa9-7c4faf30c6e3 None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:19.207638 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc137bc-d5e0-494b-8aa9-7c4faf30c6e3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:19.207941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc137bc-d5e0-494b-8aa9-7c4faf30c6e3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:19.209785 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc137bc-d5e0-494b-8aa9-7c4faf30c6e3 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:19.229331 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 325/652] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 23 16:54:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:19.234726 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc137bc-d5e0-494b-8aa9-7c4faf30c6e3 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:19.256107 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f47d822-678d-4770-a65b-78c900d8f641 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:19.270742 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:19.295992 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302e1a0-61a1-43ab-a86e-2c3b498238c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:19.296947 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302e1a0-61a1-43ab-a86e-2c3b498238c2 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:19.297610 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302e1a0-61a1-43ab-a86e-2c3b498238c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:19.297982 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c302e1a0-61a1-43ab-a86e-2c3b498238c2 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:19.298830 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302e1a0-61a1-43ab-a86e-2c3b498238c2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:19.299263 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302e1a0-61a1-43ab-a86e-2c3b498238c2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:19.300825 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c302e1a0-61a1-43ab-a86e-2c3b498238c2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:19.318113 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 328/653] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:19.318113 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302e1a0-61a1-43ab-a86e-2c3b498238c2 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:19.324302 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:19.347161 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc137bc-d5e0-494b-8aa9-7c4faf30c6e3 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:19.386408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf76d2-5fc3-4c8e-9306-f21d5014d3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:19.387215 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf76d2-5fc3-4c8e-9306-f21d5014d3ca None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:19.387587 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf76d2-5fc3-4c8e-9306-f21d5014d3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:19.387928 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7faf76d2-5fc3-4c8e-9306-f21d5014d3ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:19.388833 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf76d2-5fc3-4c8e-9306-f21d5014d3ca None admin] RBAC: Adding query filter params (role.id=5a7214dc069f45969e19c44122b378a4, scope.project.id=0595667fb2e04a3ca4716f669d6698b0, user.id=225c7b385f12445fab86b3b5606b1da1) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:19.389220 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf76d2-5fc3-4c8e-9306-f21d5014d3ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:19.390652 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7faf76d2-5fc3-4c8e-9306-f21d5014d3ca None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:19.406938 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 326/654] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:54:19 2022] GET /identity/v3/role_assignments?user.id=225c7b385f12445fab86b3b5606b1da1&scope.project.id=0595667fb2e04a3ca4716f669d6698b0&role.id=5a7214dc069f45969e19c44122b378a4 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:19.406938 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf76d2-5fc3-4c8e-9306-f21d5014d3ca None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:19.452487 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c302e1a0-61a1-43ab-a86e-2c3b498238c2 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:19.510212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59427ce-4041-4f9b-8e27-fc08746aa444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:19.511376 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59427ce-4041-4f9b-8e27-fc08746aa444 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:19.511801 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59427ce-4041-4f9b-8e27-fc08746aa444 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:19.512240 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f59427ce-4041-4f9b-8e27-fc08746aa444 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:19.522254 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59427ce-4041-4f9b-8e27-fc08746aa444 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:19.524024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f59427ce-4041-4f9b-8e27-fc08746aa444 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:19.541201 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-f59427ce-4041-4f9b-8e27-fc08746aa444 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 23 16:54:19.541201 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:19.541201 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:19.541201 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:19.541201 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:19.541201 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:19.541201 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:19.541201 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:19.541201 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:19.541201 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:19.541201 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:19.541201 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:19.541201 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:19.541201 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:19.541201 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:19.541201 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:19.542429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:19.542429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:19.542429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:19.542429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:19.542429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:19.542429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:19.542429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:19.542429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:19.542429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:19.542429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:19.542429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:19.542429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:19.542429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:19.542429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:19.542429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:19.542429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:19.542429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:19.542429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:19.543569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:19.543569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:19.543569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:19.543569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:19.543569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:19.543569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:19.543569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:19.543569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:19.543569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 23 16:54:19.543569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:19.551788 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 329/655] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 23 16:54:19 2022] GET /identity/v3/users/neutron => generated 85 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:19.551788 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59427ce-4041-4f9b-8e27-fc08746aa444 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:19.593154 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:19.660350 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7faf76d2-5fc3-4c8e-9306-f21d5014d3ca None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:19.696485 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f59427ce-4041-4f9b-8e27-fc08746aa444 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:19.711729 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831b2c4-5dd4-485f-b743-6aa8125d6011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:19.712397 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831b2c4-5dd4-485f-b743-6aa8125d6011 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:19.712678 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831b2c4-5dd4-485f-b743-6aa8125d6011 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:19.712951 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9831b2c4-5dd4-485f-b743-6aa8125d6011 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:19.728880 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:19.741629 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057b755-b5a1-4b1e-baf4-a10a8ca28eb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:19.742751 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057b755-b5a1-4b1e-baf4-a10a8ca28eb3 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:19.742751 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057b755-b5a1-4b1e-baf4-a10a8ca28eb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:19.743014 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0057b755-b5a1-4b1e-baf4-a10a8ca28eb3 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:19.756119 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0057b755-b5a1-4b1e-baf4-a10a8ca28eb3 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:19.759350 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0057b755-b5a1-4b1e-baf4-a10a8ca28eb3 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:19.778405 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-0057b755-b5a1-4b1e-baf4-a10a8ca28eb3 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 16:54:19.778405 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:19.778405 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:19.778405 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:19.778405 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:19.778405 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:19.778405 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:19.778405 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:19.778405 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:19.778405 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:19.778405 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:19.778405 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:19.778405 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:19.778405 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:19.778405 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:19.778405 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:19.780042 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:19.780042 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:19.780042 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:19.780042 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:19.780042 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:19.780042 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:19.780042 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:19.780042 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:19.780042 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:19.780042 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:19.780042 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:19.780042 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:19.780042 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:19.780042 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:19.780042 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:19.780042 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:19.780042 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:19.780042 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:19.781207 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:19.781207 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:19.781207 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:19.781207 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:19.781207 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:19.781207 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:19.781207 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:19.781207 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:19.781207 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 16:54:19.781207 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:19.784211 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 330/656] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:54:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:19.785772 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057b755-b5a1-4b1e-baf4-a10a8ca28eb3 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:19.799331 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9831b2c4-5dd4-485f-b743-6aa8125d6011 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:19.801427 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9831b2c4-5dd4-485f-b743-6aa8125d6011 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:19.894556 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:19.904591 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0057b755-b5a1-4b1e-baf4-a10a8ca28eb3 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:19.915456 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 327/657] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:54:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:54:19.915456 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831b2c4-5dd4-485f-b743-6aa8125d6011 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:19.946665 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c378c-0e62-42eb-9f18-2492d7ede094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:19.947360 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c378c-0e62-42eb-9f18-2492d7ede094 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:19.947636 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c378c-0e62-42eb-9f18-2492d7ede094 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:19.947902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c378c-0e62-42eb-9f18-2492d7ede094 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:19.948737 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c378c-0e62-42eb-9f18-2492d7ede094 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:19.949032 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c378c-0e62-42eb-9f18-2492d7ede094 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:19.950452 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314c378c-0e62-42eb-9f18-2492d7ede094 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:19.959389 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:19.972213 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 331/658] 10.222.0.13 () {56 vars in 1139 bytes} [Mon May 23 16:54:19 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:19.972213 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd23c1-924b-4049-8bf6-e5495adb058e None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:19.972213 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd23c1-924b-4049-8bf6-e5495adb058e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:19.972213 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fdd23c1-924b-4049-8bf6-e5495adb058e None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:19.972213 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 332/659] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:19 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:19.972213 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdd23c1-924b-4049-8bf6-e5495adb058e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:20.022505 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9831b2c4-5dd4-485f-b743-6aa8125d6011 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:20.059426 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17222cf7-1178-437b-bfeb-30d07b029ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:20.060106 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17222cf7-1178-437b-bfeb-30d07b029ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:20.060389 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17222cf7-1178-437b-bfeb-30d07b029ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:20.060659 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17222cf7-1178-437b-bfeb-30d07b029ea2 None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:20.061526 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17222cf7-1178-437b-bfeb-30d07b029ea2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:20.061832 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17222cf7-1178-437b-bfeb-30d07b029ea2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:20.063279 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17222cf7-1178-437b-bfeb-30d07b029ea2 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:20.081738 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 328/660] 10.222.0.13 () {56 vars in 1137 bytes} [Mon May 23 16:54:19 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:20.082233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fdd23c1-924b-4049-8bf6-e5495adb058e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:20.082800 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17222cf7-1178-437b-bfeb-30d07b029ea2 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:20.119516 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a893d104-9875-4535-a8ea-e6e540ad79e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:20.120297 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893d104-9875-4535-a8ea-e6e540ad79e9 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:20.120575 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893d104-9875-4535-a8ea-e6e540ad79e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:20.120843 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a893d104-9875-4535-a8ea-e6e540ad79e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:20.131411 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893d104-9875-4535-a8ea-e6e540ad79e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:20.133550 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a893d104-9875-4535-a8ea-e6e540ad79e9 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:20.152593 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-a893d104-9875-4535-a8ea-e6e540ad79e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:20.152593 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:20.152593 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:20.152593 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:20.152593 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:20.152593 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:20.152593 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:20.152593 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:20.152593 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:20.152593 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:20.152593 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:20.152593 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:20.152593 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:20.152593 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:20.152593 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:20.153512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:20.153512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:20.153512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:20.153512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:20.153512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:20.153512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:20.153512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:20.153512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:20.153512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:20.153512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:20.153512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:20.153512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:20.153512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:20.153512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:20.153512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:20.153512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:20.153512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:20.154412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:20.154412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:20.154412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:20.154412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:20.154412 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 333/661] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:20.155160 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd705d40-9fc6-436b-8494-2940c60613f2 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:20.155322 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd705d40-9fc6-436b-8494-2940c60613f2 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:20.155604 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd705d40-9fc6-436b-8494-2940c60613f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:20.160358 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:20.172178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-cd705d40-9fc6-436b-8494-2940c60613f2 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:20.173269 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-cd705d40-9fc6-436b-8494-2940c60613f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:20.203251 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:20.206886 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17222cf7-1178-437b-bfeb-30d07b029ea2 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:20.254963 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 334/662] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:54:20.255196 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b346906-ff11-4ee1-b961-48b17b1ad40a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:20.255333 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd705d40-9fc6-436b-8494-2940c60613f2 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:20.255421 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b346906-ff11-4ee1-b961-48b17b1ad40a None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:20.255421 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b346906-ff11-4ee1-b961-48b17b1ad40a None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:20.255421 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b346906-ff11-4ee1-b961-48b17b1ad40a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:20.325867 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b346906-ff11-4ee1-b961-48b17b1ad40a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:20.327466 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b346906-ff11-4ee1-b961-48b17b1ad40a None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:20.358268 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd705d40-9fc6-436b-8494-2940c60613f2 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:20.389569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:20.395960 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3950fb-a12b-4dbc-8330-3ba7dc3ed650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:20.396757 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3950fb-a12b-4dbc-8330-3ba7dc3ed650 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:20.397041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3950fb-a12b-4dbc-8330-3ba7dc3ed650 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:20.397319 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3950fb-a12b-4dbc-8330-3ba7dc3ed650 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:20.431982 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:20.437675 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 329/663] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:54:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:54:20.438665 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b346906-ff11-4ee1-b961-48b17b1ad40a None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:20.478016 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3950fb-a12b-4dbc-8330-3ba7dc3ed650 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:20.479632 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3950fb-a12b-4dbc-8330-3ba7dc3ed650 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:20.543817 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b346906-ff11-4ee1-b961-48b17b1ad40a None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:20.576855 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114e470-545e-4cb1-86a3-4bed309135b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:20.577526 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114e470-545e-4cb1-86a3-4bed309135b1 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:20.577821 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114e470-545e-4cb1-86a3-4bed309135b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:20.578079 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c114e470-545e-4cb1-86a3-4bed309135b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:20.578801 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114e470-545e-4cb1-86a3-4bed309135b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:20.579140 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114e470-545e-4cb1-86a3-4bed309135b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:20.580523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c114e470-545e-4cb1-86a3-4bed309135b1 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:20.585924 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 335/664] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:54:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:54:20.587206 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3950fb-a12b-4dbc-8330-3ba7dc3ed650 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:20.597222 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 330/665] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:20.598373 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114e470-545e-4cb1-86a3-4bed309135b1 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:20.623935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:20.664097 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:20.698296 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3950fb-a12b-4dbc-8330-3ba7dc3ed650 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:20.710549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c114e470-545e-4cb1-86a3-4bed309135b1 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:20.734856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21218fec-fec5-4a61-9ce4-930d94074bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:20.735698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21218fec-fec5-4a61-9ce4-930d94074bed None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:20.735979 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21218fec-fec5-4a61-9ce4-930d94074bed None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:20.736253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21218fec-fec5-4a61-9ce4-930d94074bed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:20.748292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21218fec-fec5-4a61-9ce4-930d94074bed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:20.750326 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a06c4-4a47-403a-b8cf-becce221f040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:20.751155 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a06c4-4a47-403a-b8cf-becce221f040 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:20.751436 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a06c4-4a47-403a-b8cf-becce221f040 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:20.751707 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37a06c4-4a47-403a-b8cf-becce221f040 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:20.756893 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-21218fec-fec5-4a61-9ce4-930d94074bed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:20.756893 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:20.756893 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:20.756893 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:20.756893 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:20.756893 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:20.756893 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:20.756893 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:20.756893 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:20.756893 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:20.756893 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:20.756893 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:20.756893 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:20.756893 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:20.756893 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:20.756893 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:20.757993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:20.757993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:20.757993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:20.757993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:20.757993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:20.757993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:20.757993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:20.757993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:20.757993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:20.757993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:20.757993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:20.757993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:20.757993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:20.757993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:20.757993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:20.757993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:20.759461 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:20.759461 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:20.759461 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:20.759461 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:20.759461 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:20.759461 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:20.759461 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:20.759461 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:20.759461 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:20.759461 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:20.759461 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:20.759461 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:20.766093 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 336/666] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 23 16:54:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:20.766093 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21218fec-fec5-4a61-9ce4-930d94074bed None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:20.772046 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a06c4-4a47-403a-b8cf-becce221f040 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:20.773517 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37a06c4-4a47-403a-b8cf-becce221f040 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:20.792506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-b37a06c4-4a47-403a-b8cf-becce221f040 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:20.792506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:20.792506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:20.792506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:20.792506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:20.792506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:20.792506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:20.792506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:20.792506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:20.792506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:20.792506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:20.792506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:20.792506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:20.792506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:20.792506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:20.793425 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:20.793425 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:20.793425 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:20.793425 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:20.793425 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:20.793425 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:20.793425 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:20.793425 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:20.793425 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:20.793425 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:20.793425 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:20.793425 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:20.793425 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:20.793425 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:20.793425 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:20.793425 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:20.793425 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:20.794306 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:20.794306 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:20.794306 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:20.794306 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:20.798238 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 331/667] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:20.799152 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a06c4-4a47-403a-b8cf-becce221f040 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:20.865029 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:20.874119 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21218fec-fec5-4a61-9ce4-930d94074bed None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:20.903907 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37a06c4-4a47-403a-b8cf-becce221f040 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:20.913507 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797da49-901c-474f-9410-ea48c9ff6748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:20.914421 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797da49-901c-474f-9410-ea48c9ff6748 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:20.914705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797da49-901c-474f-9410-ea48c9ff6748 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:20.915169 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2797da49-901c-474f-9410-ea48c9ff6748 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:20.917964 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:20.926935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797da49-901c-474f-9410-ea48c9ff6748 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:20.928435 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2797da49-901c-474f-9410-ea48c9ff6748 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:20.940206 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f75f3-6b4a-4b90-b47d-05f71dc0b019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:20.941156 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f75f3-6b4a-4b90-b47d-05f71dc0b019 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:20.941497 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f75f3-6b4a-4b90-b47d-05f71dc0b019 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:20.941775 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f75f3-6b4a-4b90-b47d-05f71dc0b019 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:20.946706 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-2797da49-901c-474f-9410-ea48c9ff6748 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:20.946706 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:20.946706 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:20.946706 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:20.946706 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:20.946706 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:20.946706 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:20.946706 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:20.946706 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:20.946706 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:20.946706 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:20.946706 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:20.946706 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:20.946706 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:20.946706 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:20.947592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:20.947592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:20.947592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:20.947592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:20.947592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:20.947592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:20.947592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:20.947592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:20.947592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:20.947592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:20.947592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:20.947592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:20.947592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:20.947592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:20.947592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:20.947592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:20.947592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:20.948522 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:20.948522 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:20.948522 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:20.948522 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:20.948522 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 337/668] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:20.948826 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797da49-901c-474f-9410-ea48c9ff6748 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:20.953632 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f75f3-6b4a-4b90-b47d-05f71dc0b019 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:20.955113 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f75f3-6b4a-4b90-b47d-05f71dc0b019 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:20.969054 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-4e0f75f3-6b4a-4b90-b47d-05f71dc0b019 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:20.969054 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:20.969054 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:20.969054 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:20.969054 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:20.969054 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:20.969054 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:20.969054 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:20.969054 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:20.969054 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:20.969054 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:20.969054 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:20.969054 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:20.969054 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:20.969054 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:20.970112 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:20.970112 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:20.970112 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:20.970112 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:20.970112 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:20.970112 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:20.970112 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:20.970112 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:20.970112 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:20.970112 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:20.970112 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:20.970112 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:20.970112 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:20.970112 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:20.970112 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:20.970112 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:20.970112 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:20.971850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:20.971850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:20.971850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:20.971850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:20.971850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:20.971850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:20.971850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:20.971850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:20.971850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 332/669] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:20.971850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f75f3-6b4a-4b90-b47d-05f71dc0b019 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:21.056974 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2797da49-901c-474f-9410-ea48c9ff6748 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:21.078980 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f75f3-6b4a-4b90-b47d-05f71dc0b019 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:21.092683 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e270d8c-5b03-4575-9e21-609f04c730f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:21.093357 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e270d8c-5b03-4575-9e21-609f04c730f3 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:21.093637 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e270d8c-5b03-4575-9e21-609f04c730f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:21.093910 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e270d8c-5b03-4575-9e21-609f04c730f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:21.094713 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e270d8c-5b03-4575-9e21-609f04c730f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:21.095063 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e270d8c-5b03-4575-9e21-609f04c730f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:21.096570 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e270d8c-5b03-4575-9e21-609f04c730f3 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:21.105406 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 338/670] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 23 16:54:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:21.105406 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e270d8c-5b03-4575-9e21-609f04c730f3 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:21.118049 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:21.118489 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aba01f-faad-4211-a569-bed2c32d23b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:21.119239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aba01f-faad-4211-a569-bed2c32d23b7 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:21.119560 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aba01f-faad-4211-a569-bed2c32d23b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:21.120116 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aba01f-faad-4211-a569-bed2c32d23b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:21.120934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aba01f-faad-4211-a569-bed2c32d23b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:21.121240 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aba01f-faad-4211-a569-bed2c32d23b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:21.122618 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8aba01f-faad-4211-a569-bed2c32d23b7 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:21.135961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 333/671] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:21.136788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aba01f-faad-4211-a569-bed2c32d23b7 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:21.169480 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:21.223405 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e270d8c-5b03-4575-9e21-609f04c730f3 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:21.244934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aba01f-faad-4211-a569-bed2c32d23b7 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:21.260566 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39395ece-d4d1-4040-a69e-9dc9f251f4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:21.261234 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39395ece-d4d1-4040-a69e-9dc9f251f4ae None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:21.261516 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39395ece-d4d1-4040-a69e-9dc9f251f4ae None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:21.261790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39395ece-d4d1-4040-a69e-9dc9f251f4ae None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:21.266660 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39395ece-d4d1-4040-a69e-9dc9f251f4ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:21.266660 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39395ece-d4d1-4040-a69e-9dc9f251f4ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:21.266660 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39395ece-d4d1-4040-a69e-9dc9f251f4ae None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:21.289227 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 339/672] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:21.289227 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39395ece-d4d1-4040-a69e-9dc9f251f4ae None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:21.296205 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a034b-c00e-4b3f-a82b-2128c8b6262b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:21.296878 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a034b-c00e-4b3f-a82b-2128c8b6262b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:21.297163 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a034b-c00e-4b3f-a82b-2128c8b6262b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:21.297439 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-911a034b-c00e-4b3f-a82b-2128c8b6262b None admin] PATH_INFO: `/v3/projects` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:21.298239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a034b-c00e-4b3f-a82b-2128c8b6262b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:21.298554 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a034b-c00e-4b3f-a82b-2128c8b6262b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:21.300033 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-911a034b-c00e-4b3f-a82b-2128c8b6262b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:21.312356 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 334/673] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:21.318080 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a034b-c00e-4b3f-a82b-2128c8b6262b None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:21.356442 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:21.397241 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39395ece-d4d1-4040-a69e-9dc9f251f4ae None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:21.417725 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:21.429925 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-911a034b-c00e-4b3f-a82b-2128c8b6262b None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:21.433302 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed69660-d20e-4e4b-95c2-5013be10f926 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:21.434112 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed69660-d20e-4e4b-95c2-5013be10f926 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:21.434393 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed69660-d20e-4e4b-95c2-5013be10f926 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:21.434666 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed69660-d20e-4e4b-95c2-5013be10f926 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:21.445233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed69660-d20e-4e4b-95c2-5013be10f926 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:21.446704 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed69660-d20e-4e4b-95c2-5013be10f926 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:21.464829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-bed69660-d20e-4e4b-95c2-5013be10f926 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:21.464829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:21.464829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:21.464829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:21.464829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:21.464829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:21.464829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:21.464829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:21.464829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:21.464829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:21.464829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:21.464829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:21.464829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:21.464829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:21.464829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:21.465803 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:21.465803 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:21.465803 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:21.465803 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:21.465803 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:21.465803 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:21.465803 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:21.465803 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:21.465803 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:21.465803 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:21.465803 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:21.465803 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:21.465803 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:21.465803 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:21.465803 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:21.465803 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:21.465803 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:21.466712 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:21.466712 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:21.466712 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:21.466712 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:21.466712 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 340/674] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:21.467032 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed69660-d20e-4e4b-95c2-5013be10f926 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:21.471524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b0a732-dd66-4a3d-9c8c-aab6a31483d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:21.472612 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b0a732-dd66-4a3d-9c8c-aab6a31483d9 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:21.472818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b0a732-dd66-4a3d-9c8c-aab6a31483d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:21.473064 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b0a732-dd66-4a3d-9c8c-aab6a31483d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:21.481204 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b0a732-dd66-4a3d-9c8c-aab6a31483d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:21.482660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b0a732-dd66-4a3d-9c8c-aab6a31483d9 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:21.497688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-25b0a732-dd66-4a3d-9c8c-aab6a31483d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:21.497688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:21.497688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:21.497688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:21.497688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:21.497688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:21.497688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:21.497688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:21.497688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:21.497688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:21.497688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:21.497688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:21.497688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:21.497688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:21.497688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:21.498765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:21.498765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:21.498765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:21.498765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:21.498765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:21.498765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:21.498765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:21.498765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:21.498765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:21.498765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:21.498765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:21.498765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:21.498765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:21.498765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:21.498765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:21.498765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:21.498765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:21.499696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:21.499696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:21.499696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:21.499696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:21.499696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:21.499696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:21.499696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:21.499696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:21.499696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 335/675] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:21.499696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b0a732-dd66-4a3d-9c8c-aab6a31483d9 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:21.592149 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed69660-d20e-4e4b-95c2-5013be10f926 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:21.610622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b0a732-dd66-4a3d-9c8c-aab6a31483d9 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:21.612275 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:21.634198 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d8ba2d-10a2-4b84-91a6-dba19a07d99e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:21.635278 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8ba2d-10a2-4b84-91a6-dba19a07d99e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:21.635977 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8ba2d-10a2-4b84-91a6-dba19a07d99e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:21.636261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d8ba2d-10a2-4b84-91a6-dba19a07d99e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:21.645737 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d8ba2d-10a2-4b84-91a6-dba19a07d99e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:21.646629 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc935418-66d6-434a-8913-23380b3abe51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:21.647226 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d8ba2d-10a2-4b84-91a6-dba19a07d99e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:21.647326 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc935418-66d6-434a-8913-23380b3abe51 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:21.647591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc935418-66d6-434a-8913-23380b3abe51 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:21.647868 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc935418-66d6-434a-8913-23380b3abe51 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:21.648698 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc935418-66d6-434a-8913-23380b3abe51 None admin] RBAC: Adding query filter params (role.id=5a7214dc069f45969e19c44122b378a4, scope.project.id=0595667fb2e04a3ca4716f669d6698b0, user.id=d93036329cf64a8e91edaee5b1d47b9c) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:21.649016 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc935418-66d6-434a-8913-23380b3abe51 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:21.650446 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc935418-66d6-434a-8913-23380b3abe51 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:21.656594 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 336/676] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:54:21 2022] GET /identity/v3/role_assignments?user.id=d93036329cf64a8e91edaee5b1d47b9c&scope.project.id=0595667fb2e04a3ca4716f669d6698b0&role.id=5a7214dc069f45969e19c44122b378a4 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:21.657457 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc935418-66d6-434a-8913-23380b3abe51 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:21.663462 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-59d8ba2d-10a2-4b84-91a6-dba19a07d99e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:21.663462 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:21.663462 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:21.663462 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:21.663462 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:21.663462 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:21.663462 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:21.663462 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:21.663462 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:21.663462 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:21.663462 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:21.663462 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:21.663462 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:21.663462 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:21.663462 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:21.664429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:21.664429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:21.664429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:21.664429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:21.664429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:21.664429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:21.664429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:21.664429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:21.664429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:21.664429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:21.664429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:21.664429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:21.664429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:21.664429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:21.664429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:21.664429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:21.664429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:21.665345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:21.665345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:21.665345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:21.665345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:21.665345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:21.665345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:21.665345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:21.665345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:21.665345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 341/677] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:21.666009 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d8ba2d-10a2-4b84-91a6-dba19a07d99e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:21.689702 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:21.791561 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d8ba2d-10a2-4b84-91a6-dba19a07d99e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:21.803614 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc935418-66d6-434a-8913-23380b3abe51 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:21.831317 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee098e7-b95d-44d7-9af7-3ab41a635347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:21.832082 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee098e7-b95d-44d7-9af7-3ab41a635347 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:21.832369 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee098e7-b95d-44d7-9af7-3ab41a635347 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:21.832643 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee098e7-b95d-44d7-9af7-3ab41a635347 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:21.833448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee098e7-b95d-44d7-9af7-3ab41a635347 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:21.833759 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee098e7-b95d-44d7-9af7-3ab41a635347 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:21.835269 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee098e7-b95d-44d7-9af7-3ab41a635347 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:21.841182 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24ea9df-170b-4374-a6f7-03bbe55129a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:21.841866 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ea9df-170b-4374-a6f7-03bbe55129a8 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:21.842149 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ea9df-170b-4374-a6f7-03bbe55129a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:21.842421 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24ea9df-170b-4374-a6f7-03bbe55129a8 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:21.843276 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24ea9df-170b-4374-a6f7-03bbe55129a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:21.843585 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24ea9df-170b-4374-a6f7-03bbe55129a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:21.844985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24ea9df-170b-4374-a6f7-03bbe55129a8 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:21.849298 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 342/678] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:21.855167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee098e7-b95d-44d7-9af7-3ab41a635347 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:21.858847 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 337/679] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:21.860164 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c89767-51ea-4074-886a-44f44a1310a2 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:21.860448 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c89767-51ea-4074-886a-44f44a1310a2 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:21.860725 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c89767-51ea-4074-886a-44f44a1310a2 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:21.861942 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 338/680] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:21.862631 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c89767-51ea-4074-886a-44f44a1310a2 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:21.905355 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:21.954784 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:21.973796 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee098e7-b95d-44d7-9af7-3ab41a635347 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:21.982226 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c89767-51ea-4074-886a-44f44a1310a2 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:22.008953 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfe2af-02e6-4349-ad4e-7fd8f1b1cd2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:22.009616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfe2af-02e6-4349-ad4e-7fd8f1b1cd2c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:22.009893 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfe2af-02e6-4349-ad4e-7fd8f1b1cd2c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:22.010164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62dfe2af-02e6-4349-ad4e-7fd8f1b1cd2c None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:22.010963 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfe2af-02e6-4349-ad4e-7fd8f1b1cd2c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:22.011269 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfe2af-02e6-4349-ad4e-7fd8f1b1cd2c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:22.012705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62dfe2af-02e6-4349-ad4e-7fd8f1b1cd2c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:22.018410 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bddd116-60b2-49a4-ba78-042996602bdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:22.019613 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddd116-60b2-49a4-ba78-042996602bdc None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:22.019916 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddd116-60b2-49a4-ba78-042996602bdc None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:22.020180 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bddd116-60b2-49a4-ba78-042996602bdc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:22.021414 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bddd116-60b2-49a4-ba78-042996602bdc None admin] RBAC: Adding query filter params (role.id=5a7214dc069f45969e19c44122b378a4, scope.project.id=0595667fb2e04a3ca4716f669d6698b0, user.id=fd070e7581f04f4abcbb8b6bc231a8c1) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:22.021751 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bddd116-60b2-49a4-ba78-042996602bdc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:22.023203 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bddd116-60b2-49a4-ba78-042996602bdc None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:22.028903 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 343/681] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:22.028903 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfe2af-02e6-4349-ad4e-7fd8f1b1cd2c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:22.037622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 339/682] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:54:21 2022] GET /identity/v3/role_assignments?user.id=fd070e7581f04f4abcbb8b6bc231a8c1&scope.project.id=0595667fb2e04a3ca4716f669d6698b0&role.id=5a7214dc069f45969e19c44122b378a4 => generated 608 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:22.037622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7b5bf-65e7-4df3-b3ec-b7a35c22f032 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:22.037622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7b5bf-65e7-4df3-b3ec-b7a35c22f032 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:22.037622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f7b5bf-65e7-4df3-b3ec-b7a35c22f032 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:22.054203 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-82f7b5bf-65e7-4df3-b3ec-b7a35c22f032 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:22.055357 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-82f7b5bf-65e7-4df3-b3ec-b7a35c22f032 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:22.145099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:22.149809 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62dfe2af-02e6-4349-ad4e-7fd8f1b1cd2c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:22.167126 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 340/683] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:54:22.168143 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7b5bf-65e7-4df3-b3ec-b7a35c22f032 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:22.205619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca6ffa-981b-4e7a-9bf6-2c46999e95ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:22.206721 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca6ffa-981b-4e7a-9bf6-2c46999e95ea None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:22.207031 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca6ffa-981b-4e7a-9bf6-2c46999e95ea None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:22.207300 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ca6ffa-981b-4e7a-9bf6-2c46999e95ea None admin] PATH_INFO: `/v3/users/swift` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:22.217812 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca6ffa-981b-4e7a-9bf6-2c46999e95ea None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:22.219429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ca6ffa-981b-4e7a-9bf6-2c46999e95ea None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:22.236257 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-e1ca6ffa-981b-4e7a-9bf6-2c46999e95ea None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 16:54:22.236257 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:22.236257 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:22.236257 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:22.236257 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:22.236257 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:22.236257 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:22.236257 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:22.236257 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:22.236257 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:22.236257 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:22.236257 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:22.236257 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:22.236257 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:22.236257 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:22.236257 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:22.238178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:22.238178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:22.238178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:22.238178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:22.238178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:22.238178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:22.238178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:22.238178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:22.238178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:22.238178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:22.238178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:22.238178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:22.238178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:22.238178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:22.238178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:22.238178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:22.238178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:22.238178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:22.239277 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:22.239277 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:22.239277 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:22.239277 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:22.239277 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:22.239277 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:22.239277 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:22.239277 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:22.239277 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 16:54:22.239277 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:22.239277 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 344/684] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:54:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:22.239277 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca6ffa-981b-4e7a-9bf6-2c46999e95ea None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:22.259705 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:22.298113 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f7b5bf-65e7-4df3-b3ec-b7a35c22f032 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:22.345805 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f79792-296d-4b25-b935-c6fafaebdde5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:22.346481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f79792-296d-4b25-b935-c6fafaebdde5 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:22.346758 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f79792-296d-4b25-b935-c6fafaebdde5 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:22.347073 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f79792-296d-4b25-b935-c6fafaebdde5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:22.348289 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f79792-296d-4b25-b935-c6fafaebdde5 None admin] RBAC: Adding query filter params (role.id=5a7214dc069f45969e19c44122b378a4, scope.project.id=0595667fb2e04a3ca4716f669d6698b0, user.id=22e45ab9d34c4fcbaf93be339257eca8) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:22.348852 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f79792-296d-4b25-b935-c6fafaebdde5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:22.350036 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f79792-296d-4b25-b935-c6fafaebdde5 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:22.354624 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ca6ffa-981b-4e7a-9bf6-2c46999e95ea None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:22.356288 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 341/685] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:54:22 2022] GET /identity/v3/role_assignments?user.id=22e45ab9d34c4fcbaf93be339257eca8&scope.project.id=0595667fb2e04a3ca4716f669d6698b0&role.id=5a7214dc069f45969e19c44122b378a4 => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:22.357838 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f79792-296d-4b25-b935-c6fafaebdde5 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:22.404082 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808e012-c5fd-48fe-85f4-4f2ea4958ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:22.405055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808e012-c5fd-48fe-85f4-4f2ea4958ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:22.405442 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808e012-c5fd-48fe-85f4-4f2ea4958ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:22.405782 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3808e012-c5fd-48fe-85f4-4f2ea4958ae9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:22.427641 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3808e012-c5fd-48fe-85f4-4f2ea4958ae9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:22.438181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-3808e012-c5fd-48fe-85f4-4f2ea4958ae9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:54:22.438181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:22.438181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:22.438181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:22.438181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:22.438181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:22.438181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:22.438181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:22.438181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:22.438181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:22.438181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:22.438181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:22.438181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:22.438181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:22.438181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:22.438181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:22.439401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:22.439401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:22.439401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:22.439401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:22.439401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:22.439401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:22.439401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:22.439401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:22.439401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:22.439401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:22.439401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:22.439401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:22.439401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:22.439401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:22.439401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:22.439401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:22.440408 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:22.440408 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:22.440408 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:22.440408 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:22.440408 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:22.440408 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:22.440408 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:22.440408 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:22.440408 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:22.440408 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:22.440408 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:54:22.440408 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:22.450822 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 345/686] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:54:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:22.459743 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808e012-c5fd-48fe-85f4-4f2ea4958ae9 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:22.501514 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:22.556851 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f79792-296d-4b25-b935-c6fafaebdde5 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:22.601691 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431dd1f-c994-4955-a054-2f49dd7efd07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:22.602750 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431dd1f-c994-4955-a054-2f49dd7efd07 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:22.603064 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431dd1f-c994-4955-a054-2f49dd7efd07 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:22.603337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7431dd1f-c994-4955-a054-2f49dd7efd07 None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:22.604171 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431dd1f-c994-4955-a054-2f49dd7efd07 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:22.605634 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431dd1f-c994-4955-a054-2f49dd7efd07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:22.607748 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7431dd1f-c994-4955-a054-2f49dd7efd07 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:22.610318 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:22.638694 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 342/687] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 23 16:54:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:22.641051 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431dd1f-c994-4955-a054-2f49dd7efd07 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:22.648646 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3808e012-c5fd-48fe-85f4-4f2ea4958ae9 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:22.692487 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd465b76-8d94-45a9-9af9-459cdcdc4d46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:22.692487 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd465b76-8d94-45a9-9af9-459cdcdc4d46 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:22.692783 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd465b76-8d94-45a9-9af9-459cdcdc4d46 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:22.692943 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd465b76-8d94-45a9-9af9-459cdcdc4d46 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:22.693873 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd465b76-8d94-45a9-9af9-459cdcdc4d46 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:22.694247 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd465b76-8d94-45a9-9af9-459cdcdc4d46 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:22.697119 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd465b76-8d94-45a9-9af9-459cdcdc4d46 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:22.709284 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 346/688] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:54:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:22.718028 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd465b76-8d94-45a9-9af9-459cdcdc4d46 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:22.773613 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7431dd1f-c994-4955-a054-2f49dd7efd07 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:22.806702 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:22.811210 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b81d4f-9899-4103-8eed-6004664762e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:22.811980 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b81d4f-9899-4103-8eed-6004664762e2 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:22.812573 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b81d4f-9899-4103-8eed-6004664762e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:22.812710 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b81d4f-9899-4103-8eed-6004664762e2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:22.829105 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd465b76-8d94-45a9-9af9-459cdcdc4d46 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:22.864647 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf15770-318c-4828-a8f5-83bd2c36ba96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:22.865726 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf15770-318c-4828-a8f5-83bd2c36ba96 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:22.866145 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf15770-318c-4828-a8f5-83bd2c36ba96 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:22.866683 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf15770-318c-4828-a8f5-83bd2c36ba96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:22.876885 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:22.878196 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf15770-318c-4828-a8f5-83bd2c36ba96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:22.880214 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf15770-318c-4828-a8f5-83bd2c36ba96 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:22.889499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b81d4f-9899-4103-8eed-6004664762e2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:22.890878 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b81d4f-9899-4103-8eed-6004664762e2 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:22.899335 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-edf15770-318c-4828-a8f5-83bd2c36ba96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:22.899335 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:22.899335 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:22.899335 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:22.899335 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:22.899335 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:22.899335 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:22.899335 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:22.899335 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:22.899335 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:22.899335 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:22.899335 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:22.899335 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:22.899335 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:22.899335 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:22.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:22.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:22.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:22.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:22.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:22.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:22.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:22.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:22.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:22.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:22.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:22.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:22.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:22.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:22.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:22.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:22.900239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:22.901109 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:22.901109 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:22.901109 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:22.901109 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:22.903349 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 347/689] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:22.909474 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf15770-318c-4828-a8f5-83bd2c36ba96 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:23.073980 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 343/690] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:54:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:54:23.075713 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b81d4f-9899-4103-8eed-6004664762e2 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:23.097815 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf15770-318c-4828-a8f5-83bd2c36ba96 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:23.164702 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:23.170935 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:23.181815 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b81d4f-9899-4103-8eed-6004664762e2 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:23.183690 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7aa1c-433e-4305-85c0-3600dde260c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:23.184380 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7aa1c-433e-4305-85c0-3600dde260c9 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:23.184665 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7aa1c-433e-4305-85c0-3600dde260c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:23.184943 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a7aa1c-433e-4305-85c0-3600dde260c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:23.185682 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7aa1c-433e-4305-85c0-3600dde260c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:23.185993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7aa1c-433e-4305-85c0-3600dde260c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:23.187428 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a7aa1c-433e-4305-85c0-3600dde260c9 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:23.214329 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 348/691] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:23.220594 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7aa1c-433e-4305-85c0-3600dde260c9 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:23.224671 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a06952-cdb8-493b-8826-3ae64691095f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:23.225550 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a06952-cdb8-493b-8826-3ae64691095f None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:23.225835 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a06952-cdb8-493b-8826-3ae64691095f None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:23.226106 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a06952-cdb8-493b-8826-3ae64691095f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:23.245438 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a06952-cdb8-493b-8826-3ae64691095f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:23.246901 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a06952-cdb8-493b-8826-3ae64691095f None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:23.284652 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-c1a06952-cdb8-493b-8826-3ae64691095f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:23.284652 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:23.284652 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:23.284652 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:23.284652 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:23.284652 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:23.284652 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:23.284652 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:23.284652 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:23.284652 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:23.284652 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:23.284652 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:23.284652 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:23.284652 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:23.284652 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:23.286008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:23.286008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:23.286008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:23.286008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:23.286008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:23.286008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:23.286008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:23.286008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:23.286008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:23.286008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:23.286008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:23.286008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:23.286008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:23.286008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:23.286008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:23.286008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:23.286008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:23.287008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:23.287008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:23.287008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:23.287008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:23.287008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 344/692] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:23.296102 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a06952-cdb8-493b-8826-3ae64691095f None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:23.405244 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a7aa1c-433e-4305-85c0-3600dde260c9 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:23.412062 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a06952-cdb8-493b-8826-3ae64691095f None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:23.452727 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0c493-ff09-4723-9321-209f42785b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:23.453321 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0c493-ff09-4723-9321-209f42785b75 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:23.453603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0c493-ff09-4723-9321-209f42785b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:23.453875 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af0c493-ff09-4723-9321-209f42785b75 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:23.454617 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af0c493-ff09-4723-9321-209f42785b75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:23.455017 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af0c493-ff09-4723-9321-209f42785b75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:23.456667 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af0c493-ff09-4723-9321-209f42785b75 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:23.470861 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 345/693] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:23.476718 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0c493-ff09-4723-9321-209f42785b75 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:23.482163 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:23.495733 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9e9c6-bec2-4d58-bba5-7250ac4a2ae6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:23.496556 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9e9c6-bec2-4d58-bba5-7250ac4a2ae6 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:23.496831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9e9c6-bec2-4d58-bba5-7250ac4a2ae6 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:23.497093 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa9e9c6-bec2-4d58-bba5-7250ac4a2ae6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:23.512141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9e9c6-bec2-4d58-bba5-7250ac4a2ae6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:23.513630 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa9e9c6-bec2-4d58-bba5-7250ac4a2ae6 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:23.555128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-baa9e9c6-bec2-4d58-bba5-7250ac4a2ae6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 16:54:23.555128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:23.555128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:23.555128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:23.555128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:23.555128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:23.555128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:23.555128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:23.555128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:23.555128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:23.555128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:23.555128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:23.555128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:23.555128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:23.555128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:23.555128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:23.556145 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:23.556145 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:23.556145 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:23.556145 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:23.556145 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:23.556145 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:23.556145 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:23.556145 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:23.556145 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:23.556145 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:23.556145 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:23.556145 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:23.556145 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:23.556145 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:23.556145 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:23.556145 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:23.556145 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:23.556145 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:23.557077 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:23.557077 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:23.557077 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:23.557077 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:23.557077 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:23.557077 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:23.557077 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:23.557077 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:23.557077 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 16:54:23.557077 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:23.557077 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 349/694] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 23 16:54:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 337 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:54:23.565819 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9e9c6-bec2-4d58-bba5-7250ac4a2ae6 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:23.591758 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af0c493-ff09-4723-9321-209f42785b75 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:23.627148 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0afc9-32a6-44f8-bb8b-51264b7c17d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:23.627953 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0afc9-32a6-44f8-bb8b-51264b7c17d6 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:23.628230 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0afc9-32a6-44f8-bb8b-51264b7c17d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:23.628604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e0afc9-32a6-44f8-bb8b-51264b7c17d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:23.636166 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e0afc9-32a6-44f8-bb8b-51264b7c17d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:23.637621 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e0afc9-32a6-44f8-bb8b-51264b7c17d6 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:23.652103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-e0e0afc9-32a6-44f8-bb8b-51264b7c17d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:23.652103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:23.652103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:23.652103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:23.652103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:23.652103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:23.652103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:23.652103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:23.652103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:23.652103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:23.652103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:23.652103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:23.652103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:23.652103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:23.652103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:23.654207 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:23.654207 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:23.654207 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:23.654207 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:23.654207 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:23.654207 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:23.654207 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:23.654207 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:23.654207 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:23.654207 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:23.654207 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:23.654207 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:23.654207 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:23.654207 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:23.654207 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:23.654207 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:23.654207 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:23.656308 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:23.656308 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:23.656308 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:23.656308 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:23.656308 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:23.656308 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:23.656308 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:23.656308 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:23.662942 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 346/695] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:23.662942 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0afc9-32a6-44f8-bb8b-51264b7c17d6 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:23.699095 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:23.725353 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:23.765025 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e0afc9-32a6-44f8-bb8b-51264b7c17d6 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:23.799169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6ca65-a720-48f4-a575-e297c6cfb82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:23.799845 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6ca65-a720-48f4-a575-e297c6cfb82e None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:23.800134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6ca65-a720-48f4-a575-e297c6cfb82e None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:23.800584 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c6ca65-a720-48f4-a575-e297c6cfb82e None admin] PATH_INFO: `/v3/projects` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:23.801464 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6ca65-a720-48f4-a575-e297c6cfb82e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:23.801782 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6ca65-a720-48f4-a575-e297c6cfb82e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:23.803251 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c6ca65-a720-48f4-a575-e297c6cfb82e None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:23.808885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa9e9c6-bec2-4d58-bba5-7250ac4a2ae6 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:23.820534 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 347/696] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:23.821758 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6ca65-a720-48f4-a575-e297c6cfb82e None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:23.879323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311164f5-563f-49c9-9f59-de662666ab1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:23.880040 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311164f5-563f-49c9-9f59-de662666ab1d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:23.880321 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311164f5-563f-49c9-9f59-de662666ab1d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:23.880594 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311164f5-563f-49c9-9f59-de662666ab1d None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:23.881438 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311164f5-563f-49c9-9f59-de662666ab1d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:23.881745 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311164f5-563f-49c9-9f59-de662666ab1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:23.883537 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-311164f5-563f-49c9-9f59-de662666ab1d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:23.919591 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 350/697] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 23 16:54:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:23.939999 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311164f5-563f-49c9-9f59-de662666ab1d None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:23.982537 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c6ca65-a720-48f4-a575-e297c6cfb82e None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:24.038001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:24.061219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267a747b-c83c-42a7-b56f-b74d3e7b2ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:24.061935 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a747b-c83c-42a7-b56f-b74d3e7b2ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:24.062227 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a747b-c83c-42a7-b56f-b74d3e7b2ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:24.062505 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267a747b-c83c-42a7-b56f-b74d3e7b2ee9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:24.063380 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a747b-c83c-42a7-b56f-b74d3e7b2ee9 None admin] RBAC: Adding query filter params (role.id=5a7214dc069f45969e19c44122b378a4, scope.project.id=0595667fb2e04a3ca4716f669d6698b0, user.id=6546217667054abb8eb07076995c1b04) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:24.063704 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a747b-c83c-42a7-b56f-b74d3e7b2ee9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:24.066399 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267a747b-c83c-42a7-b56f-b74d3e7b2ee9 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:24.082327 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 348/698] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:54:23 2022] GET /identity/v3/role_assignments?user.id=6546217667054abb8eb07076995c1b04&scope.project.id=0595667fb2e04a3ca4716f669d6698b0&role.id=5a7214dc069f45969e19c44122b378a4 => generated 608 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:24.118731 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-311164f5-563f-49c9-9f59-de662666ab1d None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:24.135854 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:24.177567 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07456d6d-82ef-478a-91c5-fbb3d739d112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:24.178254 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07456d6d-82ef-478a-91c5-fbb3d739d112 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:24.178539 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07456d6d-82ef-478a-91c5-fbb3d739d112 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:24.178815 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07456d6d-82ef-478a-91c5-fbb3d739d112 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:24.263456 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07456d6d-82ef-478a-91c5-fbb3d739d112 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:24.264964 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07456d6d-82ef-478a-91c5-fbb3d739d112 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:24.318679 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34dff1-bf1a-47ba-837f-29ab973b7cbb None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:24.319692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34dff1-bf1a-47ba-837f-29ab973b7cbb None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:24.320169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb34dff1-bf1a-47ba-837f-29ab973b7cbb None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:24.321969 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 349/699] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:54:24.328461 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb819f72-efa3-4a7e-b42e-6000ebef2d2d None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:24.329020 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb819f72-efa3-4a7e-b42e-6000ebef2d2d None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:24.329514 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb819f72-efa3-4a7e-b42e-6000ebef2d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:24.355310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-eb819f72-efa3-4a7e-b42e-6000ebef2d2d None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:24.356935 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-eb819f72-efa3-4a7e-b42e-6000ebef2d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:24.371522 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 351/700] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:54:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 16:54:24.377819 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07456d6d-82ef-478a-91c5-fbb3d739d112 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:24.392970 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:24.459688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 350/701] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:54:24.486091 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb819f72-efa3-4a7e-b42e-6000ebef2d2d None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:24.519515 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07456d6d-82ef-478a-91c5-fbb3d739d112 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:24.555973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e603eb-78e3-47c7-a00e-83ab934e0fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:24.557209 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e603eb-78e3-47c7-a00e-83ab934e0fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:24.557833 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e603eb-78e3-47c7-a00e-83ab934e0fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:24.558308 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e603eb-78e3-47c7-a00e-83ab934e0fa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:24.570160 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e603eb-78e3-47c7-a00e-83ab934e0fa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:24.572037 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e603eb-78e3-47c7-a00e-83ab934e0fa9 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:24.592672 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-75e603eb-78e3-47c7-a00e-83ab934e0fa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:24.592672 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:24.592672 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:24.592672 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:24.592672 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:24.592672 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:24.592672 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:24.592672 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:24.592672 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:24.592672 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:24.592672 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:24.592672 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:24.592672 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:24.592672 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:24.592672 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:24.593618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:24.593618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:24.593618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:24.593618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:24.593618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:24.593618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:24.593618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:24.593618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:24.593618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:24.593618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:24.593618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:24.593618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:24.593618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:24.593618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:24.593618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:24.593618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:24.593618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:24.594516 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:24.594516 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:24.594516 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:24.594516 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:24.594746 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb819f72-efa3-4a7e-b42e-6000ebef2d2d None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:24.597241 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 352/702] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:54:24.603209 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e603eb-78e3-47c7-a00e-83ab934e0fa9 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:24.613353 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:24.632706 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff342323-b780-46d9-8d8f-64e1cde689c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:24.633740 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff342323-b780-46d9-8d8f-64e1cde689c9 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:24.634031 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff342323-b780-46d9-8d8f-64e1cde689c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:24.634291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff342323-b780-46d9-8d8f-64e1cde689c9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:24.645057 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff342323-b780-46d9-8d8f-64e1cde689c9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:24.646525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff342323-b780-46d9-8d8f-64e1cde689c9 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:24.665945 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-ff342323-b780-46d9-8d8f-64e1cde689c9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:24.665945 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:24.665945 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:24.665945 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:24.665945 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:24.665945 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:24.665945 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:24.665945 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:24.665945 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:24.665945 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:24.665945 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:24.665945 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:24.665945 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:24.665945 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:24.665945 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:24.666955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:24.666955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:24.666955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:24.666955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:24.666955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:24.666955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:24.666955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:24.666955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:24.666955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:24.666955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:24.666955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:24.666955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:24.666955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:24.666955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:24.666955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:24.666955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:24.666955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:24.667879 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:24.667879 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:24.667879 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:24.667879 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:24.667879 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 351/703] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:24.673225 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff342323-b780-46d9-8d8f-64e1cde689c9 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:24.679575 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:24.721373 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e603eb-78e3-47c7-a00e-83ab934e0fa9 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:24.764548 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a38d8c9-b7d1-43f2-b0a3-811b6bae69d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:24.765224 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38d8c9-b7d1-43f2-b0a3-811b6bae69d8 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:24.765504 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38d8c9-b7d1-43f2-b0a3-811b6bae69d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:24.765920 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a38d8c9-b7d1-43f2-b0a3-811b6bae69d8 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:24.766654 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a38d8c9-b7d1-43f2-b0a3-811b6bae69d8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:24.766994 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a38d8c9-b7d1-43f2-b0a3-811b6bae69d8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:24.768532 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a38d8c9-b7d1-43f2-b0a3-811b6bae69d8 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:24.778269 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff342323-b780-46d9-8d8f-64e1cde689c9 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:24.786946 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 353/704] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:24.788611 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ca996-9e5a-4f02-b032-8832ff0f8ec9 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:24.788904 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ca996-9e5a-4f02-b032-8832ff0f8ec9 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:24.789280 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ca996-9e5a-4f02-b032-8832ff0f8ec9 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:24.794766 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 354/705] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:24.796781 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092c65f-6717-4f04-9d6b-e0b70261f9f3 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:24.797071 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092c65f-6717-4f04-9d6b-e0b70261f9f3 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:24.797345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2092c65f-6717-4f04-9d6b-e0b70261f9f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:24.815185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-2092c65f-6717-4f04-9d6b-e0b70261f9f3 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:24.816188 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f696370-fb33-42d6-8c62-154151652149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:24.816526 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-2092c65f-6717-4f04-9d6b-e0b70261f9f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:24.816867 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f696370-fb33-42d6-8c62-154151652149 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:24.817151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f696370-fb33-42d6-8c62-154151652149 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:24.817427 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f696370-fb33-42d6-8c62-154151652149 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:24.818161 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f696370-fb33-42d6-8c62-154151652149 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:24.818466 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f696370-fb33-42d6-8c62-154151652149 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:24.819873 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f696370-fb33-42d6-8c62-154151652149 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:24.832645 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 352/706] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:24.833513 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f696370-fb33-42d6-8c62-154151652149 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:24.868794 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:24.895106 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 355/707] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:54:24.896306 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092c65f-6717-4f04-9d6b-e0b70261f9f3 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:24.949972 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f696370-fb33-42d6-8c62-154151652149 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:24.971152 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:24.987179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0494b24-c01d-47c7-8527-48882bbee87e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:24.988000 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0494b24-c01d-47c7-8527-48882bbee87e None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:24.988286 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0494b24-c01d-47c7-8527-48882bbee87e None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:24.988722 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0494b24-c01d-47c7-8527-48882bbee87e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:24.996380 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0494b24-c01d-47c7-8527-48882bbee87e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:24.997832 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0494b24-c01d-47c7-8527-48882bbee87e None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:25.012496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-b0494b24-c01d-47c7-8527-48882bbee87e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:25.012496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:25.012496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:25.012496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:25.012496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:25.012496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:25.012496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:25.012496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:25.012496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:25.012496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:25.012496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:25.012496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:25.012496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:25.012496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:25.012496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:25.013403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:25.013403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:25.013403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:25.013403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:25.013403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:25.013403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:25.013403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:25.013403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:25.013403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:25.013403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:25.013403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:25.013403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:25.013403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:25.013403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:25.013403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:25.013403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:25.013403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:25.014294 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:25.014294 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:25.014294 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:25.014294 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:25.014294 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:25.014294 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:25.014294 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:25.014294 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:25.014294 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 353/708] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:25.014777 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0494b24-c01d-47c7-8527-48882bbee87e None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:25.033169 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2092c65f-6717-4f04-9d6b-e0b70261f9f3 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:25.082024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4307f66-78a2-47c2-8879-e16cc66c32f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:25.083136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4307f66-78a2-47c2-8879-e16cc66c32f9 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:25.083424 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4307f66-78a2-47c2-8879-e16cc66c32f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:25.083699 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4307f66-78a2-47c2-8879-e16cc66c32f9 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:25.084698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4307f66-78a2-47c2-8879-e16cc66c32f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:25.086404 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4307f66-78a2-47c2-8879-e16cc66c32f9 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:25.118988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 356/709] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 23 16:54:24 2022] POST /identity/v3/users => generated 254 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:54:25.120238 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4307f66-78a2-47c2-8879-e16cc66c32f9 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:25.164875 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:25.186238 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0494b24-c01d-47c7-8527-48882bbee87e None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:25.253713 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd390775-a241-4a49-95aa-04b3f1480120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:25.254341 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd390775-a241-4a49-95aa-04b3f1480120 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:25.254622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd390775-a241-4a49-95aa-04b3f1480120 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:25.254895 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd390775-a241-4a49-95aa-04b3f1480120 None admin] PATH_INFO: `/v3/services/network` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:25.255601 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd390775-a241-4a49-95aa-04b3f1480120 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:25.257002 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd390775-a241-4a49-95aa-04b3f1480120 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:25.272485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-cd390775-a241-4a49-95aa-04b3f1480120 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 23 16:54:25.272485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:25.272485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:54:25.272485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:54:25.272485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:54:25.272485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:54:25.272485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:54:25.272485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:25.272485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 16:54:25.272485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:25.272485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:54:25.272485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:25.272485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:25.272485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:25.272485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:25.272485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:25.272485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:25.273630 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:25.273630 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:25.273630 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:25.273630 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:25.273630 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:25.273630 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:25.273630 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:54:25.273630 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:54:25.273630 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:54:25.273630 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:54:25.273630 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:25.273630 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:25.273630 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:25.273630 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:25.273630 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:25.273630 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:25.274526 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:25.274526 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:25.274526 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:25.274526 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:25.274526 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:25.274526 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:25.274526 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:25.274526 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:25.274526 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:25.274526 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:25.274526 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:54:25.274526 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:25.274526 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 16:54:25.274526 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:25.274526 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 354/710] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:25 2022] GET /identity/v3/services/network => generated 88 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:25.276006 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa18817-a59c-41de-94d5-0d90bd5baa07 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:25.276006 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa18817-a59c-41de-94d5-0d90bd5baa07 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:25.276166 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa18817-a59c-41de-94d5-0d90bd5baa07 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:25.281761 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 355/711] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:25.282836 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa18817-a59c-41de-94d5-0d90bd5baa07 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:25.307599 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4307f66-78a2-47c2-8879-e16cc66c32f9 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:25.318659 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:25.348721 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ccf557-b652-4dce-8e74-354eecdeb023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:25.349805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccf557-b652-4dce-8e74-354eecdeb023 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:25.350302 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccf557-b652-4dce-8e74-354eecdeb023 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:25.350646 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccf557-b652-4dce-8e74-354eecdeb023 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:25.351495 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ccf557-b652-4dce-8e74-354eecdeb023 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:25.352630 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ccf557-b652-4dce-8e74-354eecdeb023 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:25.353293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ccf557-b652-4dce-8e74-354eecdeb023 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:25.367254 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 357/712] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:25.368583 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2b4d3-3a31-4793-aa79-726d567cd7a8 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:25.368870 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2b4d3-3a31-4793-aa79-726d567cd7a8 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:25.369147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d2b4d3-3a31-4793-aa79-726d567cd7a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:25.387113 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-46d2b4d3-3a31-4793-aa79-726d567cd7a8 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:25.387487 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-46d2b4d3-3a31-4793-aa79-726d567cd7a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:25.404460 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa18817-a59c-41de-94d5-0d90bd5baa07 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:25.448060 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997288e6-2d89-4941-b7b5-80746e273e6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:25.448843 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997288e6-2d89-4941-b7b5-80746e273e6b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:25.449244 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997288e6-2d89-4941-b7b5-80746e273e6b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:25.449574 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997288e6-2d89-4941-b7b5-80746e273e6b None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:25.450411 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997288e6-2d89-4941-b7b5-80746e273e6b None admin] RBAC: Adding query filter params (name=network) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:25.450770 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997288e6-2d89-4941-b7b5-80746e273e6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:25.452256 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997288e6-2d89-4941-b7b5-80746e273e6b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:25.457385 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 356/713] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 23 16:54:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:25.458568 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997288e6-2d89-4941-b7b5-80746e273e6b None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:25.476154 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 358/714] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:54:25.477040 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2b4d3-3a31-4793-aa79-726d567cd7a8 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:25.504081 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:25.601814 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997288e6-2d89-4941-b7b5-80746e273e6b None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:25.604127 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:25.607731 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d2b4d3-3a31-4793-aa79-726d567cd7a8 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:25.644320 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685c47c-d43e-4791-ac1a-f87c9ea97171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:25.645953 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685c47c-d43e-4791-ac1a-f87c9ea97171 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:25.646421 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685c47c-d43e-4791-ac1a-f87c9ea97171 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:25.646841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0685c47c-d43e-4791-ac1a-f87c9ea97171 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:25.647926 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685c47c-d43e-4791-ac1a-f87c9ea97171 None admin] RBAC: Adding query filter params (role.id=17bbe7cc1c3641509de5e9dca8534102, scope.project.id=0595667fb2e04a3ca4716f669d6698b0, user.id=225c7b385f12445fab86b3b5606b1da1) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:25.648404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685c47c-d43e-4791-ac1a-f87c9ea97171 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:25.650053 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0685c47c-d43e-4791-ac1a-f87c9ea97171 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:25.657613 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 357/715] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:54:25 2022] GET /identity/v3/role_assignments?user.id=225c7b385f12445fab86b3b5606b1da1&scope.project.id=0595667fb2e04a3ca4716f669d6698b0&role.id=17bbe7cc1c3641509de5e9dca8534102 => generated 259 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:25.658826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685c47c-d43e-4791-ac1a-f87c9ea97171 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:25.767392 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:25.787218 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0685c47c-d43e-4791-ac1a-f87c9ea97171 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:25.827731 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ec6d5-2ab1-46fe-ac7f-de25c6743f0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:25.828853 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ec6d5-2ab1-46fe-ac7f-de25c6743f0d None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:25.829280 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ec6d5-2ab1-46fe-ac7f-de25c6743f0d None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:25.829695 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e4ec6d5-2ab1-46fe-ac7f-de25c6743f0d None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:25.830501 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ec6d5-2ab1-46fe-ac7f-de25c6743f0d None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:25.833457 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e4ec6d5-2ab1-46fe-ac7f-de25c6743f0d None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:25.847987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-7e4ec6d5-2ab1-46fe-ac7f-de25c6743f0d None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 16:54:25.847987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:25.847987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:54:25.847987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:54:25.847987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:54:25.847987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:54:25.847987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:54:25.847987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:25.847987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 16:54:25.847987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:25.847987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:54:25.847987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:25.847987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:25.847987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:25.847987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:25.847987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:25.849788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:25.849788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:25.849788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:25.849788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:25.849788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:25.849788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:25.849788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:25.849788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:54:25.849788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:54:25.849788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:54:25.849788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:54:25.849788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:25.849788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:25.849788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:25.849788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:25.849788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:25.850835 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:25.850835 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:25.850835 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:25.850835 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:25.850835 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:25.850835 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:25.850835 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:25.850835 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:25.850835 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:25.850835 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:25.850835 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:25.850835 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:54:25.850835 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:25.850835 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 16:54:25.850835 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:25.853519 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 358/716] 10.222.0.13 () {56 vars in 1129 bytes} [Mon May 23 16:54:25 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:25.859691 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ec6d5-2ab1-46fe-ac7f-de25c6743f0d None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:25.863928 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152bdc5-bae6-4834-abac-22b4f5161002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:25.864281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152bdc5-bae6-4834-abac-22b4f5161002 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:25.864567 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152bdc5-bae6-4834-abac-22b4f5161002 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:25.864848 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6152bdc5-bae6-4834-abac-22b4f5161002 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:25.866098 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152bdc5-bae6-4834-abac-22b4f5161002 None admin] RBAC: Adding query filter params (type=network) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:25.866098 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152bdc5-bae6-4834-abac-22b4f5161002 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:25.867458 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6152bdc5-bae6-4834-abac-22b4f5161002 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:25.882610 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 359/717] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 23 16:54:25 2022] GET /identity/v3/services?type=network => generated 124 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:25.989255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e4ec6d5-2ab1-46fe-ac7f-de25c6743f0d None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:26.029050 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba1fe46-08e8-4f78-b717-c4748a07ae65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:26.030000 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1fe46-08e8-4f78-b717-c4748a07ae65 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:26.030427 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1fe46-08e8-4f78-b717-c4748a07ae65 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:26.030847 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba1fe46-08e8-4f78-b717-c4748a07ae65 None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:26.031757 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba1fe46-08e8-4f78-b717-c4748a07ae65 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:26.032228 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba1fe46-08e8-4f78-b717-c4748a07ae65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:26.033813 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba1fe46-08e8-4f78-b717-c4748a07ae65 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:26.040130 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 359/718] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 23 16:54:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:26.045940 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152bdc5-bae6-4834-abac-22b4f5161002 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:26.189429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6152bdc5-bae6-4834-abac-22b4f5161002 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:26.225962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b096661e-49ac-4bd5-b137-055260c3b73e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:26.226644 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096661e-49ac-4bd5-b137-055260c3b73e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:26.226936 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096661e-49ac-4bd5-b137-055260c3b73e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:26.227220 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b096661e-49ac-4bd5-b137-055260c3b73e None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:26.227931 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096661e-49ac-4bd5-b137-055260c3b73e None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:26.228235 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096661e-49ac-4bd5-b137-055260c3b73e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:26.229614 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b096661e-49ac-4bd5-b137-055260c3b73e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:26.236440 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 360/719] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 23 16:54:26 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:26.889022 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ae807-11ad-49e0-acf9-a969f9ab2463 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:26.890069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ae807-11ad-49e0-acf9-a969f9ab2463 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:26.890540 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ae807-11ad-49e0-acf9-a969f9ab2463 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:26.892394 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 360/720] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:26.898094 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac45900-de36-4d7f-a4aa-4abc29c7df0c None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:26.898625 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac45900-de36-4d7f-a4aa-4abc29c7df0c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:26.899131 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac45900-de36-4d7f-a4aa-4abc29c7df0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:26.919974 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-6ac45900-de36-4d7f-a4aa-4abc29c7df0c None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:26.921447 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-6ac45900-de36-4d7f-a4aa-4abc29c7df0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:26.950959 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:27.019694 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 361/721] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:54:27.051113 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ae807-11ad-49e0-acf9-a969f9ab2463 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:27.069002 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:27.183455 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ae807-11ad-49e0-acf9-a969f9ab2463 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:27.228767 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a996e-6cb7-4444-a13a-812c148cd98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:27.229557 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a996e-6cb7-4444-a13a-812c148cd98d None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:27.229838 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a996e-6cb7-4444-a13a-812c148cd98d None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:27.230108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648a996e-6cb7-4444-a13a-812c148cd98d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:27.252024 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648a996e-6cb7-4444-a13a-812c148cd98d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:27.260188 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-648a996e-6cb7-4444-a13a-812c148cd98d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:54:27.260188 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:27.260188 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:27.260188 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:27.260188 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:27.260188 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:27.260188 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:27.260188 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:27.260188 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:27.260188 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:27.260188 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:27.260188 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:27.260188 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:27.260188 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:27.260188 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:27.260188 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:27.261133 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:27.261133 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:27.261133 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:27.261133 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:27.261133 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:27.261133 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:27.261133 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:27.261133 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:27.261133 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:27.261133 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:27.261133 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:27.261133 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:27.261133 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:27.261133 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:27.261133 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:27.261133 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:27.262403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:27.262403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:27.262403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:27.262403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:27.262403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:27.262403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:27.262403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:27.262403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:27.262403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:27.262403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:27.262403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:54:27.262403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:27.262403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 361/722] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:54:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:27.269338 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac45900-de36-4d7f-a4aa-4abc29c7df0c None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:27.397115 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac45900-de36-4d7f-a4aa-4abc29c7df0c None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:27.433858 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4426fa-edc9-4cca-acd3-410ec5a102a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:27.434905 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4426fa-edc9-4cca-acd3-410ec5a102a0 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:27.435417 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4426fa-edc9-4cca-acd3-410ec5a102a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:27.435917 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4426fa-edc9-4cca-acd3-410ec5a102a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:27.436854 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4426fa-edc9-4cca-acd3-410ec5a102a0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:27.437352 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4426fa-edc9-4cca-acd3-410ec5a102a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:27.438958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4426fa-edc9-4cca-acd3-410ec5a102a0 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:27.452122 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 362/723] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:54:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:27.453853 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a996e-6cb7-4444-a13a-812c148cd98d None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:27.554354 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:27.596597 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648a996e-6cb7-4444-a13a-812c148cd98d None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:27.651644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37fbbd9-696d-49b3-a477-42ef3fa7b0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:27.652466 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fbbd9-696d-49b3-a477-42ef3fa7b0db None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:27.652747 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fbbd9-696d-49b3-a477-42ef3fa7b0db None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:27.653019 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37fbbd9-696d-49b3-a477-42ef3fa7b0db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:27.667964 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37fbbd9-696d-49b3-a477-42ef3fa7b0db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:27.669424 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37fbbd9-696d-49b3-a477-42ef3fa7b0db None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:27.687463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-d37fbbd9-696d-49b3-a477-42ef3fa7b0db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:27.687463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:27.687463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:27.687463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:27.687463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:27.687463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:27.687463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:27.687463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:27.687463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:27.687463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:27.687463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:27.687463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:27.687463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:27.687463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:27.687463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:27.688378 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:27.688378 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:27.688378 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:27.688378 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:27.688378 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:27.688378 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:27.688378 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:27.688378 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:27.688378 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:27.688378 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:27.688378 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:27.688378 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:27.688378 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:27.688378 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:27.688378 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:27.688378 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:27.688378 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:27.689360 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:27.689360 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:27.689360 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:27.689360 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:27.693491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 362/724] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:27.695047 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4426fa-edc9-4cca-acd3-410ec5a102a0 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:27.716592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:27.814598 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4426fa-edc9-4cca-acd3-410ec5a102a0 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:27.852424 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b38bda-5ab6-40dc-b791-7f450dd1cca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:27.853096 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b38bda-5ab6-40dc-b791-7f450dd1cca5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:27.853375 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b38bda-5ab6-40dc-b791-7f450dd1cca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:27.853647 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b38bda-5ab6-40dc-b791-7f450dd1cca5 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:27.854367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b38bda-5ab6-40dc-b791-7f450dd1cca5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:27.854670 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b38bda-5ab6-40dc-b791-7f450dd1cca5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:27.856191 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b38bda-5ab6-40dc-b791-7f450dd1cca5 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:27.872949 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 363/725] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:27.874416 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37fbbd9-696d-49b3-a477-42ef3fa7b0db None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:27.992839 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a8cf9-17da-45f1-9357-cfe40a0778a7 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:27.993139 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a8cf9-17da-45f1-9357-cfe40a0778a7 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:27.993320 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2a8cf9-17da-45f1-9357-cfe40a0778a7 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:27.996136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 364/726] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:27.997359 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37fbbd9-696d-49b3-a477-42ef3fa7b0db None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:28.000351 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e656f5-76b8-4907-8a76-cba744b63dee None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:28.000642 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e656f5-76b8-4907-8a76-cba744b63dee None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:28.000922 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e656f5-76b8-4907-8a76-cba744b63dee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:28.019066 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-d5e656f5-76b8-4907-8a76-cba744b63dee None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:28.022833 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-d5e656f5-76b8-4907-8a76-cba744b63dee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:28.038962 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:28.058664 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500910e-7f2f-4e6c-a996-3fdace1778e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:28.059506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500910e-7f2f-4e6c-a996-3fdace1778e7 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:28.059788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500910e-7f2f-4e6c-a996-3fdace1778e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:28.060058 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3500910e-7f2f-4e6c-a996-3fdace1778e7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:28.077662 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500910e-7f2f-4e6c-a996-3fdace1778e7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:28.079308 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3500910e-7f2f-4e6c-a996-3fdace1778e7 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:28.108220 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-3500910e-7f2f-4e6c-a996-3fdace1778e7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 16:54:28.108220 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:28.108220 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:28.108220 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:28.108220 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:28.108220 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:28.108220 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:28.108220 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:28.108220 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:28.108220 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:28.108220 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:28.108220 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:28.108220 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:28.108220 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:28.108220 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:28.108220 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:28.109293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:28.109293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:28.109293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:28.109293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:28.109293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:28.109293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:28.109293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:28.109293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:28.109293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:28.109293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:28.109293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:28.109293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:28.109293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:28.109293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:28.109293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:28.109293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:28.109293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:28.109293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:28.110255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:28.110255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:28.110255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:28.110255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:28.110255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:28.110255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:28.110255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:28.110255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:28.110255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 16:54:28.110255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:28.116942 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 363/727] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:54:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:28.120791 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500910e-7f2f-4e6c-a996-3fdace1778e7 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:28.172613 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 365/728] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 16:54:28.197550 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e656f5-76b8-4907-8a76-cba744b63dee None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:28.212642 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:28.333933 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3500910e-7f2f-4e6c-a996-3fdace1778e7 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:28.387309 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e656f5-76b8-4907-8a76-cba744b63dee None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:28.399274 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3feb548f-2ce3-424e-bcf4-8a449135f003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:28.400068 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb548f-2ce3-424e-bcf4-8a449135f003 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:28.400354 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb548f-2ce3-424e-bcf4-8a449135f003 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:28.400633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3feb548f-2ce3-424e-bcf4-8a449135f003 None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:28.401476 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb548f-2ce3-424e-bcf4-8a449135f003 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:28.401789 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb548f-2ce3-424e-bcf4-8a449135f003 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:28.403480 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3feb548f-2ce3-424e-bcf4-8a449135f003 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:28.422460 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 364/729] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 23 16:54:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:28.423929 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a90e5-6fe9-4d68-84dd-51e0fa98ff6d None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:28.424126 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a90e5-6fe9-4d68-84dd-51e0fa98ff6d None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:28.424404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2a90e5-6fe9-4d68-84dd-51e0fa98ff6d None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:28.429803 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 365/730] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:28.430613 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a90e5-6fe9-4d68-84dd-51e0fa98ff6d None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:28.433888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3387e0c3-09ce-4c5f-94c2-2e1cd71dfe71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:28.434980 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387e0c3-09ce-4c5f-94c2-2e1cd71dfe71 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:28.435487 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387e0c3-09ce-4c5f-94c2-2e1cd71dfe71 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:28.435951 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3387e0c3-09ce-4c5f-94c2-2e1cd71dfe71 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:28.447131 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3387e0c3-09ce-4c5f-94c2-2e1cd71dfe71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:28.455181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-3387e0c3-09ce-4c5f-94c2-2e1cd71dfe71 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:28.455181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:28.455181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:28.455181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:28.455181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:28.455181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:28.455181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:28.455181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:28.455181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:28.455181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:28.455181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:28.455181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:28.455181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:28.455181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:28.455181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:28.455181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:28.456129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:28.456129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:28.456129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:28.456129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:28.456129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:28.456129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:28.456129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:28.456129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:28.456129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:28.456129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:28.456129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:28.456129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:28.456129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:28.456129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:28.456129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:28.456129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:28.457221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:28.457221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:28.457221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:28.457221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:28.457221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:28.457221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:28.457221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:28.457221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:28.457221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:28.457221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:28.457221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:28.457221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:28.464462 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 366/731] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 23 16:54:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:54:28.466345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71d7fd-62b8-4a79-a177-f93c8c0f80b1 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:28.466841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71d7fd-62b8-4a79-a177-f93c8c0f80b1 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:28.467334 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b71d7fd-62b8-4a79-a177-f93c8c0f80b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:28.486724 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-8b71d7fd-62b8-4a79-a177-f93c8c0f80b1 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:28.488425 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8b71d7fd-62b8-4a79-a177-f93c8c0f80b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:28.490431 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:28.561392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:28.573283 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 367/732] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:54:28.573283 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71d7fd-62b8-4a79-a177-f93c8c0f80b1 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:28.575247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2a90e5-6fe9-4d68-84dd-51e0fa98ff6d None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:28.627064 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7241075d-db41-4527-bdce-a079f5ca1a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:28.627893 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241075d-db41-4527-bdce-a079f5ca1a46 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:28.628096 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241075d-db41-4527-bdce-a079f5ca1a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:28.628376 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7241075d-db41-4527-bdce-a079f5ca1a46 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:28.703680 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7241075d-db41-4527-bdce-a079f5ca1a46 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:28.705129 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7241075d-db41-4527-bdce-a079f5ca1a46 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:28.719446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b71d7fd-62b8-4a79-a177-f93c8c0f80b1 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:28.747902 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:28.756100 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62a06b-5266-4080-b1bc-2b19bcf50f5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:28.756779 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62a06b-5266-4080-b1bc-2b19bcf50f5c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:28.757164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62a06b-5266-4080-b1bc-2b19bcf50f5c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:28.757433 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d62a06b-5266-4080-b1bc-2b19bcf50f5c None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:28.758487 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62a06b-5266-4080-b1bc-2b19bcf50f5c None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:28.758792 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62a06b-5266-4080-b1bc-2b19bcf50f5c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:28.760192 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d62a06b-5266-4080-b1bc-2b19bcf50f5c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:28.773396 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 368/733] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 23 16:54:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:28.773396 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62a06b-5266-4080-b1bc-2b19bcf50f5c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:28.815884 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 366/734] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:54:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 16:54:28.817227 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c29728-295c-40bb-9b35-0cc43c14af83 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:28.817512 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c29728-295c-40bb-9b35-0cc43c14af83 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:28.817789 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c29728-295c-40bb-9b35-0cc43c14af83 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:28.822176 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 367/735] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:28.823568 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c29728-295c-40bb-9b35-0cc43c14af83 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:28.838467 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:28.886410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d62a06b-5266-4080-b1bc-2b19bcf50f5c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:28.923711 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33e911-236c-4482-9fa9-ef3282b20990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:28.924533 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33e911-236c-4482-9fa9-ef3282b20990 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:28.924805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33e911-236c-4482-9fa9-ef3282b20990 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:28.925200 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33e911-236c-4482-9fa9-ef3282b20990 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:28.933596 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c29728-295c-40bb-9b35-0cc43c14af83 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:28.935652 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a33e911-236c-4482-9fa9-ef3282b20990 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:28.943470 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-6a33e911-236c-4482-9fa9-ef3282b20990 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:54:28.943470 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:28.943470 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:28.943470 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:28.943470 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:28.943470 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:28.943470 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:28.943470 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:28.943470 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:28.943470 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:28.943470 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:28.943470 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:28.943470 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:28.943470 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:28.943470 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:28.943470 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:28.944410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:28.944410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:28.944410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:28.944410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:28.944410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:28.944410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:28.944410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:28.944410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:28.944410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:28.944410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:28.944410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:28.944410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:28.944410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:28.944410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:28.944410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:28.944410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:28.945270 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:28.945270 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:28.945270 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:28.945270 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:28.945270 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:28.945270 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:28.945270 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:28.945270 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:28.945270 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:28.945270 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:28.945270 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:54:28.945270 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:28.945270 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 369/736] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:54:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:28.946044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33e911-236c-4482-9fa9-ef3282b20990 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:28.977478 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6673c6-104b-4e6e-bc14-eb3fee51a6af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:28.978390 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6673c6-104b-4e6e-bc14-eb3fee51a6af None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:28.978741 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6673c6-104b-4e6e-bc14-eb3fee51a6af None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:28.979138 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6673c6-104b-4e6e-bc14-eb3fee51a6af None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:28.988503 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:28.991324 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6673c6-104b-4e6e-bc14-eb3fee51a6af None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:28.993002 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6673c6-104b-4e6e-bc14-eb3fee51a6af None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:29.014579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-cb6673c6-104b-4e6e-bc14-eb3fee51a6af None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:29.014579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:29.014579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:29.014579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:29.014579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:29.014579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:29.014579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:29.014579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:29.014579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:29.014579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:29.014579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:29.014579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:29.014579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:29.014579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:29.014579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:29.016037 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:29.016037 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:29.016037 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:29.016037 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:29.016037 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:29.016037 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:29.016037 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:29.016037 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:29.016037 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:29.016037 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:29.016037 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:29.016037 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:29.016037 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:29.016037 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:29.016037 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:29.016037 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:29.016037 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:29.016928 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:29.016928 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:29.016928 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:29.016928 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:29.016928 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 368/737] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:29.017730 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7a99a-cbd2-4085-b829-01a096836644 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:29.017730 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7a99a-cbd2-4085-b829-01a096836644 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:29.017866 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb7a99a-cbd2-4085-b829-01a096836644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:29.035638 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-3bb7a99a-cbd2-4085-b829-01a096836644 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:29.036825 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-3bb7a99a-cbd2-4085-b829-01a096836644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:29.057139 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33e911-236c-4482-9fa9-ef3282b20990 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:29.077216 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:29.092144 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643445ce-47b5-4442-8ecf-e69df20e7168 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:29.092942 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643445ce-47b5-4442-8ecf-e69df20e7168 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:29.093251 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643445ce-47b5-4442-8ecf-e69df20e7168 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:29.093511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-643445ce-47b5-4442-8ecf-e69df20e7168 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:29.104150 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 369/738] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:54:29.104276 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643445ce-47b5-4442-8ecf-e69df20e7168 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:29.104991 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7a99a-cbd2-4085-b829-01a096836644 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:29.105645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-643445ce-47b5-4442-8ecf-e69df20e7168 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:29.122717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-643445ce-47b5-4442-8ecf-e69df20e7168 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:29.122717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:29.122717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:29.122717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:29.122717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:29.122717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:29.122717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:29.122717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:29.122717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:29.122717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:29.122717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:29.122717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:29.122717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:29.122717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:29.122717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:29.123817 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:29.123817 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:29.123817 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:29.123817 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:29.123817 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:29.123817 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:29.123817 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:29.123817 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:29.123817 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:29.123817 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:29.123817 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:29.123817 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:29.123817 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:29.123817 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:29.123817 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:29.123817 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:29.123817 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:29.129660 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:29.129660 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:29.129660 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:29.129660 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:29.129660 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 370/739] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:29.129660 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643445ce-47b5-4442-8ecf-e69df20e7168 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:29.217339 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb7a99a-cbd2-4085-b829-01a096836644 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:29.227416 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-643445ce-47b5-4442-8ecf-e69df20e7168 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:29.239777 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:29.251882 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d584646-e7aa-4252-b016-374109a0bb5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:29.252630 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d584646-e7aa-4252-b016-374109a0bb5f None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:29.252974 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d584646-e7aa-4252-b016-374109a0bb5f None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:29.253493 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d584646-e7aa-4252-b016-374109a0bb5f None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:29.254332 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d584646-e7aa-4252-b016-374109a0bb5f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:29.254698 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d584646-e7aa-4252-b016-374109a0bb5f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:29.256162 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d584646-e7aa-4252-b016-374109a0bb5f None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:29.261936 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8b587-9b09-4345-9a38-decdac2e2891 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:29.262606 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8b587-9b09-4345-9a38-decdac2e2891 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:29.262931 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8b587-9b09-4345-9a38-decdac2e2891 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:29.263202 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa8b587-9b09-4345-9a38-decdac2e2891 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:29.263948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8b587-9b09-4345-9a38-decdac2e2891 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:29.264252 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8b587-9b09-4345-9a38-decdac2e2891 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:29.264948 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 370/740] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:54:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:29.265617 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa8b587-9b09-4345-9a38-decdac2e2891 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:29.266184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2740aee-c1c3-496b-9d07-f3aae9e3f709 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:29.266469 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2740aee-c1c3-496b-9d07-f3aae9e3f709 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:29.266751 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2740aee-c1c3-496b-9d07-f3aae9e3f709 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:29.268002 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 371/741] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:29 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:29.268714 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2740aee-c1c3-496b-9d07-f3aae9e3f709 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:29.280560 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 371/742] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:29.281385 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8b587-9b09-4345-9a38-decdac2e2891 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:29.310686 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:29.373150 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2740aee-c1c3-496b-9d07-f3aae9e3f709 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:29.381693 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa8b587-9b09-4345-9a38-decdac2e2891 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:29.409271 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bcff1d-8f10-40ae-b3d1-c3ae25c89dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:29.410053 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcff1d-8f10-40ae-b3d1-c3ae25c89dd2 None admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:29.410401 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcff1d-8f10-40ae-b3d1-c3ae25c89dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:29.410695 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bcff1d-8f10-40ae-b3d1-c3ae25c89dd2 None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:29.411548 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bcff1d-8f10-40ae-b3d1-c3ae25c89dd2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:29.412920 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bcff1d-8f10-40ae-b3d1-c3ae25c89dd2 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:29.415932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5254acf-6d10-4d82-be75-16038bf27e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:29.416596 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5254acf-6d10-4d82-be75-16038bf27e05 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:29.416898 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5254acf-6d10-4d82-be75-16038bf27e05 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:29.417147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5254acf-6d10-4d82-be75-16038bf27e05 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:29.417951 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5254acf-6d10-4d82-be75-16038bf27e05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:29.418571 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5254acf-6d10-4d82-be75-16038bf27e05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:29.420396 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5254acf-6d10-4d82-be75-16038bf27e05 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:29.428289 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 372/743] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 23 16:54:29 2022] POST /identity/v3/services => generated 240 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:54:29.429110 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bcff1d-8f10-40ae-b3d1-c3ae25c89dd2 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:29.433140 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 372/744] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:29.434527 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd890d6e-a035-4da7-8119-d28edcfad2f6 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:29.434820 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd890d6e-a035-4da7-8119-d28edcfad2f6 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:29.435139 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd890d6e-a035-4da7-8119-d28edcfad2f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:29.453490 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-dd890d6e-a035-4da7-8119-d28edcfad2f6 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:29.454709 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-dd890d6e-a035-4da7-8119-d28edcfad2f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:29.496935 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:29.555136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 373/745] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:29 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 16:54:29.555136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd890d6e-a035-4da7-8119-d28edcfad2f6 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:29.562538 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bcff1d-8f10-40ae-b3d1-c3ae25c89dd2 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:29.585912 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:29.606718 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324ab8d-ae76-46d8-b804-0bc74a03f2ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:29.607576 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324ab8d-ae76-46d8-b804-0bc74a03f2ef None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:29.607864 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324ab8d-ae76-46d8-b804-0bc74a03f2ef None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:29.608142 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c324ab8d-ae76-46d8-b804-0bc74a03f2ef None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:29.620656 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324ab8d-ae76-46d8-b804-0bc74a03f2ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:29.622188 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c324ab8d-ae76-46d8-b804-0bc74a03f2ef None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:29.640408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-c324ab8d-ae76-46d8-b804-0bc74a03f2ef None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:29.640408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:29.640408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:29.640408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:29.640408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:29.640408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:29.640408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:29.640408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:29.640408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:29.640408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:29.640408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:29.640408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:29.640408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:29.640408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:29.640408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:29.641314 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:29.641314 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:29.641314 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:29.641314 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:29.641314 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:29.641314 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:29.641314 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:29.641314 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:29.641314 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:29.641314 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:29.641314 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:29.641314 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:29.641314 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:29.641314 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:29.641314 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:29.641314 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:29.641314 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:29.642178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:29.642178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:29.642178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:29.642178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:29.642178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 373/746] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:29.642457 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324ab8d-ae76-46d8-b804-0bc74a03f2ef None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:29.685838 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd890d6e-a035-4da7-8119-d28edcfad2f6 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:29.721642 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2985d00-4504-4919-b715-2d540ec0acaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:29.722901 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2985d00-4504-4919-b715-2d540ec0acaa None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:29.723328 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2985d00-4504-4919-b715-2d540ec0acaa None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:29.723607 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2985d00-4504-4919-b715-2d540ec0acaa None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:29.732015 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2985d00-4504-4919-b715-2d540ec0acaa None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:29.733513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2985d00-4504-4919-b715-2d540ec0acaa None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:29.747012 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:29.749441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-a2985d00-4504-4919-b715-2d540ec0acaa None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 16:54:29.749441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:29.749441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:29.749441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:29.749441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:29.749441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:29.749441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:29.749441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:29.749441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:29.749441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:29.749441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:29.749441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:29.749441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:29.749441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:29.749441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:29.749441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:29.758674 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:29.758674 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:29.758674 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:29.758674 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:29.758674 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:29.758674 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:29.758674 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:29.758674 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:29.758674 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:29.758674 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:29.758674 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:29.758674 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:29.758674 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:29.758674 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:29.758674 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:29.758674 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:29.758674 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:29.758674 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:29.760313 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:29.760313 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:29.760313 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:29.760313 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:29.760313 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:29.760313 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:29.760313 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:29.760313 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:29.760313 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 16:54:29.760313 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:29.760313 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 374/747] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 23 16:54:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:29.760313 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2985d00-4504-4919-b715-2d540ec0acaa None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:29.761335 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c324ab8d-ae76-46d8-b804-0bc74a03f2ef None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:29.800476 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67098e5-0cfd-49b1-a375-d624fe38731f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:29.801296 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67098e5-0cfd-49b1-a375-d624fe38731f None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:29.801595 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67098e5-0cfd-49b1-a375-d624fe38731f None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:29.801836 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67098e5-0cfd-49b1-a375-d624fe38731f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:29.809128 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67098e5-0cfd-49b1-a375-d624fe38731f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:29.810566 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c67098e5-0cfd-49b1-a375-d624fe38731f None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:29.824644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-c67098e5-0cfd-49b1-a375-d624fe38731f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:29.824644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:29.824644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:29.824644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:29.824644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:29.824644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:29.824644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:29.824644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:29.824644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:29.824644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:29.824644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:29.824644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:29.824644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:29.824644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:29.824644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:29.825604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:29.825604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:29.825604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:29.825604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:29.825604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:29.825604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:29.825604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:29.825604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:29.825604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:29.825604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:29.825604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:29.825604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:29.825604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:29.825604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:29.825604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:29.825604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:29.825604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:29.831316 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:29.831316 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:29.831316 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:29.831316 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:29.831316 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:29.831316 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:29.831316 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:29.831316 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:29.831316 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 374/748] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:29.831316 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67098e5-0cfd-49b1-a375-d624fe38731f None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:29.853715 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:29.877668 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2985d00-4504-4919-b715-2d540ec0acaa None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:29.915807 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9bcd07-9863-4632-9efd-118859cc7996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:29.916467 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bcd07-9863-4632-9efd-118859cc7996 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:29.916742 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bcd07-9863-4632-9efd-118859cc7996 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:29.917007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9bcd07-9863-4632-9efd-118859cc7996 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:29.917717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9bcd07-9863-4632-9efd-118859cc7996 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:29.919113 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9bcd07-9863-4632-9efd-118859cc7996 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:29.934819 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67098e5-0cfd-49b1-a375-d624fe38731f None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:29.935027 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 375/749] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 23 16:54:29 2022] POST /identity/v3/services => generated 251 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:54:29.935585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9bcd07-9863-4632-9efd-118859cc7996 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:29.990866 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8b1e7-9a03-4753-8b75-d39cf56d8cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:29.991512 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8b1e7-9a03-4753-8b75-d39cf56d8cbc None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:29.991847 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8b1e7-9a03-4753-8b75-d39cf56d8cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:29.992126 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8b1e7-9a03-4753-8b75-d39cf56d8cbc None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:29.992908 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8b1e7-9a03-4753-8b75-d39cf56d8cbc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:29.993222 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8b1e7-9a03-4753-8b75-d39cf56d8cbc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:29.994706 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f8b1e7-9a03-4753-8b75-d39cf56d8cbc None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:30.017012 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 375/750] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:30.018093 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8b1e7-9a03-4753-8b75-d39cf56d8cbc None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:30.042937 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:30.076070 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9bcd07-9863-4632-9efd-118859cc7996 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:30.116156 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a65bad-e4bd-4018-9133-9bb02672936f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:30.116915 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a65bad-e4bd-4018-9133-9bb02672936f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:30.117284 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a65bad-e4bd-4018-9133-9bb02672936f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:30.117581 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a65bad-e4bd-4018-9133-9bb02672936f None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:30.118432 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a65bad-e4bd-4018-9133-9bb02672936f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:30.118742 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a65bad-e4bd-4018-9133-9bb02672936f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:30.120223 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a65bad-e4bd-4018-9133-9bb02672936f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:30.130631 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:30.146740 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 376/751] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 23 16:54:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:30.147967 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a65bad-e4bd-4018-9133-9bb02672936f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:30.164629 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8b1e7-9a03-4753-8b75-d39cf56d8cbc None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:30.211685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2547ed-3ffb-44ce-884e-6b71b519fec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:30.213371 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2547ed-3ffb-44ce-884e-6b71b519fec7 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:30.214098 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2547ed-3ffb-44ce-884e-6b71b519fec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:30.214565 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2547ed-3ffb-44ce-884e-6b71b519fec7 None admin] PATH_INFO: `/v3/projects` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:30.216205 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2547ed-3ffb-44ce-884e-6b71b519fec7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:30.217001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2547ed-3ffb-44ce-884e-6b71b519fec7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:30.219291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2547ed-3ffb-44ce-884e-6b71b519fec7 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:30.233421 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 376/752] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:30.238949 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2547ed-3ffb-44ce-884e-6b71b519fec7 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:30.277438 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a65bad-e4bd-4018-9133-9bb02672936f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:30.311071 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:30.317934 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278c050-03a5-4cf8-8a6a-a91263b3588a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:30.318937 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278c050-03a5-4cf8-8a6a-a91263b3588a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:30.319357 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278c050-03a5-4cf8-8a6a-a91263b3588a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:30.319786 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5278c050-03a5-4cf8-8a6a-a91263b3588a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:30.327611 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278c050-03a5-4cf8-8a6a-a91263b3588a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:30.329337 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5278c050-03a5-4cf8-8a6a-a91263b3588a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:30.343934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2547ed-3ffb-44ce-884e-6b71b519fec7 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:30.345007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-5278c050-03a5-4cf8-8a6a-a91263b3588a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 16:54:30.345007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:30.345007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:30.345007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:30.345007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:30.345007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:30.345007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:30.345007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:30.345007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:30.345007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:30.345007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:30.345007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:30.345007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:30.345007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:30.345007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:30.345007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:30.345932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:30.345932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:30.345932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:30.345932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:30.345932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:30.345932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:30.345932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:30.345932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:30.345932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:30.345932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:30.345932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:30.345932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:30.345932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:30.345932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:30.345932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:30.345932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:30.345932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:30.345932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:30.346889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:30.346889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:30.346889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:30.346889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:30.346889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:30.346889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:30.346889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:30.346889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:30.346889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 16:54:30.346889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:30.349654 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 377/753] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 23 16:54:30 2022] GET /identity/v3/users/nova => generated 82 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:30.350645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278c050-03a5-4cf8-8a6a-a91263b3588a None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:30.385727 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b510f-85f6-4ebd-9eae-376e157eea03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:30.386432 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b510f-85f6-4ebd-9eae-376e157eea03 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:30.386712 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b510f-85f6-4ebd-9eae-376e157eea03 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:30.387060 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221b510f-85f6-4ebd-9eae-376e157eea03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:30.411195 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:30.458731 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b510f-85f6-4ebd-9eae-376e157eea03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:30.460213 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221b510f-85f6-4ebd-9eae-376e157eea03 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:30.462618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5278c050-03a5-4cf8-8a6a-a91263b3588a None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:30.507089 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ca06f-c504-4d73-bc4d-ebcc8fedcfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:30.508336 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ca06f-c504-4d73-bc4d-ebcc8fedcfd4 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:30.508760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ca06f-c504-4d73-bc4d-ebcc8fedcfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:30.509171 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d6ca06f-c504-4d73-bc4d-ebcc8fedcfd4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:30.510156 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ca06f-c504-4d73-bc4d-ebcc8fedcfd4 None admin] RBAC: Adding query filter params (role.id=17bbe7cc1c3641509de5e9dca8534102, scope.project.id=0595667fb2e04a3ca4716f669d6698b0, user.id=6546217667054abb8eb07076995c1b04) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:30.510616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ca06f-c504-4d73-bc4d-ebcc8fedcfd4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:30.512222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d6ca06f-c504-4d73-bc4d-ebcc8fedcfd4 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:30.523294 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 378/754] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:54:30 2022] GET /identity/v3/role_assignments?user.id=6546217667054abb8eb07076995c1b04&scope.project.id=0595667fb2e04a3ca4716f669d6698b0&role.id=17bbe7cc1c3641509de5e9dca8534102 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:30.524445 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ca06f-c504-4d73-bc4d-ebcc8fedcfd4 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:30.539418 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:30.573514 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 377/755] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:54:30 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 16:54:30.575059 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b510f-85f6-4ebd-9eae-376e157eea03 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:30.687955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221b510f-85f6-4ebd-9eae-376e157eea03 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:30.717607 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d6ca06f-c504-4d73-bc4d-ebcc8fedcfd4 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:30.732757 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177acbe3-fdf2-4485-9482-5b56c7c6f104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:30.733579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177acbe3-fdf2-4485-9482-5b56c7c6f104 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:30.733859 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177acbe3-fdf2-4485-9482-5b56c7c6f104 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:30.734249 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177acbe3-fdf2-4485-9482-5b56c7c6f104 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:30.753601 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177acbe3-fdf2-4485-9482-5b56c7c6f104 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:30.755019 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177acbe3-fdf2-4485-9482-5b56c7c6f104 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:30.767329 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:30.786901 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-177acbe3-fdf2-4485-9482-5b56c7c6f104 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:30.786901 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:30.786901 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:30.786901 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:30.786901 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:30.786901 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:30.786901 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:30.786901 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:30.786901 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:30.786901 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:30.786901 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:30.786901 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:30.786901 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:30.786901 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:30.786901 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:30.787870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:30.787870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:30.787870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:30.787870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:30.787870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:30.787870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:30.787870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:30.787870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:30.787870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:30.787870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:30.787870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:30.787870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:30.787870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:30.787870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:30.787870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:30.787870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:30.787870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:30.788863 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:30.788863 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:30.788863 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:30.788863 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:30.795038 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 378/756] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:30.801239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177acbe3-fdf2-4485-9482-5b56c7c6f104 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:30.804461 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fcd1b2-6b83-4655-9497-6f655fd6b13e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:30.805938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fcd1b2-6b83-4655-9497-6f655fd6b13e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:30.805938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fcd1b2-6b83-4655-9497-6f655fd6b13e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:30.805938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fcd1b2-6b83-4655-9497-6f655fd6b13e None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:30.806653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fcd1b2-6b83-4655-9497-6f655fd6b13e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:30.806983 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fcd1b2-6b83-4655-9497-6f655fd6b13e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:30.808457 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fcd1b2-6b83-4655-9497-6f655fd6b13e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:30.856624 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 379/757] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 23 16:54:30 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:30.857383 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fcd1b2-6b83-4655-9497-6f655fd6b13e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:30.891283 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:31.054373 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fcd1b2-6b83-4655-9497-6f655fd6b13e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:31.099801 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d9db2-915d-4459-a922-6bd519680376 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:31.100617 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d9db2-915d-4459-a922-6bd519680376 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:31.100892 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d9db2-915d-4459-a922-6bd519680376 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:31.101154 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3d9db2-915d-4459-a922-6bd519680376 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:31.134201 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177acbe3-fdf2-4485-9482-5b56c7c6f104 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:31.143850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d9db2-915d-4459-a922-6bd519680376 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:31.145415 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3d9db2-915d-4459-a922-6bd519680376 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:31.190167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-0e3d9db2-915d-4459-a922-6bd519680376 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:31.190167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:31.190167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:31.190167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:31.190167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:31.190167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:31.190167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:31.190167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:31.190167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:31.190167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:31.190167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:31.190167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:31.190167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:31.190167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:31.190167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:31.191971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:31.191971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:31.191971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:31.191971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:31.191971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:31.191971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:31.191971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:31.191971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:31.191971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:31.191971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:31.191971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:31.191971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:31.191971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:31.191971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:31.191971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:31.191971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:31.191971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:31.192861 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:31.192861 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:31.192861 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:31.192861 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:31.196070 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 380/758] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:31.197257 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d9db2-915d-4459-a922-6bd519680376 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:31.259332 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe87167-2a18-48ed-a31c-e753672c0eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:31.260019 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe87167-2a18-48ed-a31c-e753672c0eb6 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:31.260302 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe87167-2a18-48ed-a31c-e753672c0eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:31.260580 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe87167-2a18-48ed-a31c-e753672c0eb6 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:31.261315 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe87167-2a18-48ed-a31c-e753672c0eb6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:31.266938 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe87167-2a18-48ed-a31c-e753672c0eb6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:31.268362 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe87167-2a18-48ed-a31c-e753672c0eb6 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:31.294879 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:31.310966 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 379/759] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 515 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:31.312959 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe87167-2a18-48ed-a31c-e753672c0eb6 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:31.475281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:31.506767 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe87167-2a18-48ed-a31c-e753672c0eb6 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:31.528266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3d9db2-915d-4459-a922-6bd519680376 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:31.558467 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd2afc-6df3-4382-a7eb-047e22fdc5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:31.559703 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd2afc-6df3-4382-a7eb-047e22fdc5cf None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:31.560189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd2afc-6df3-4382-a7eb-047e22fdc5cf None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:31.560681 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cd2afc-6df3-4382-a7eb-047e22fdc5cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:31.569151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd2afc-6df3-4382-a7eb-047e22fdc5cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:31.570989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cd2afc-6df3-4382-a7eb-047e22fdc5cf None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:31.586788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-a5cd2afc-6df3-4382-a7eb-047e22fdc5cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:31.586788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:31.586788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:31.586788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:31.586788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:31.586788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:31.586788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:31.586788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:31.586788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:31.586788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:31.586788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:31.586788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:31.586788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:31.586788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:31.586788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:31.587961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:31.587961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:31.587961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:31.587961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:31.587961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:31.587961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:31.587961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:31.587961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:31.587961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:31.587961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:31.587961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:31.587961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:31.587961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:31.587961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:31.587961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:31.587961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:31.587961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:31.588859 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:31.588859 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:31.588859 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:31.588859 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:31.588859 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:31.588859 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:31.588859 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:31.588859 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:31.595039 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 380/760] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:31.597462 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd2afc-6df3-4382-a7eb-047e22fdc5cf None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:31.635198 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a07001-7d8b-49ad-a461-6ac368c83f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:31.637751 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a07001-7d8b-49ad-a461-6ac368c83f21 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:31.638178 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a07001-7d8b-49ad-a461-6ac368c83f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:31.638439 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a07001-7d8b-49ad-a461-6ac368c83f21 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:31.639208 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a07001-7d8b-49ad-a461-6ac368c83f21 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:31.639521 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a07001-7d8b-49ad-a461-6ac368c83f21 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:31.642108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a07001-7d8b-49ad-a461-6ac368c83f21 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:31.666446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 381/761] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 469 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:31.671324 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a07001-7d8b-49ad-a461-6ac368c83f21 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:31.786704 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cd2afc-6df3-4382-a7eb-047e22fdc5cf None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:31.808915 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:31.822278 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a07001-7d8b-49ad-a461-6ac368c83f21 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:31.829290 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04db43ac-f7b9-42ae-8fb6-1c220e7de861 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:31.830206 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db43ac-f7b9-42ae-8fb6-1c220e7de861 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:31.830681 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db43ac-f7b9-42ae-8fb6-1c220e7de861 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:31.831754 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04db43ac-f7b9-42ae-8fb6-1c220e7de861 None admin] PATH_INFO: `/v3/projects` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:31.832768 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db43ac-f7b9-42ae-8fb6-1c220e7de861 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:31.833280 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db43ac-f7b9-42ae-8fb6-1c220e7de861 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:31.835582 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04db43ac-f7b9-42ae-8fb6-1c220e7de861 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:31.851284 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 381/762] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:31.853016 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642aba1-6711-4b59-8d91-ef0851b2b644 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:31.853598 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642aba1-6711-4b59-8d91-ef0851b2b644 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:31.854062 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2642aba1-6711-4b59-8d91-ef0851b2b644 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:31.860095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 382/763] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:31.860095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642aba1-6711-4b59-8d91-ef0851b2b644 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:31.872287 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae1a932-75c2-49f9-bc07-1be25b603b61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:31.873476 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1a932-75c2-49f9-bc07-1be25b603b61 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:31.873979 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1a932-75c2-49f9-bc07-1be25b603b61 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:31.874451 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae1a932-75c2-49f9-bc07-1be25b603b61 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:31.882592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae1a932-75c2-49f9-bc07-1be25b603b61 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:31.884657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae1a932-75c2-49f9-bc07-1be25b603b61 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:31.895289 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:31.900334 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-dae1a932-75c2-49f9-bc07-1be25b603b61 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:31.900334 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:31.900334 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:31.900334 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:31.900334 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:31.900334 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:31.900334 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:31.900334 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:31.900334 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:31.900334 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:31.900334 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:31.900334 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:31.900334 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:31.900334 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:31.900334 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:31.901212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:31.901212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:31.901212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:31.901212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:31.901212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:31.901212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:31.901212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:31.901212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:31.901212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:31.901212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:31.901212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:31.901212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:31.901212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:31.901212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:31.901212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:31.901212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:31.901212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:31.902158 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:31.902158 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:31.902158 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:31.902158 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:31.902158 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:31.902158 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:31.902158 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:31.902158 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:31.909182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 382/764] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:31.910754 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4098158-1b2e-4075-a09c-94d0d9f7f689 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:31.911275 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4098158-1b2e-4075-a09c-94d0d9f7f689 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:31.911753 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4098158-1b2e-4075-a09c-94d0d9f7f689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:31.930495 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-e4098158-1b2e-4075-a09c-94d0d9f7f689 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:31.932156 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e4098158-1b2e-4075-a09c-94d0d9f7f689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:31.976025 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2642aba1-6711-4b59-8d91-ef0851b2b644 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:32.019073 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24227e04-fd14-4515-81b5-686f73bc8643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:32.019073 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24227e04-fd14-4515-81b5-686f73bc8643 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:32.019073 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24227e04-fd14-4515-81b5-686f73bc8643 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:32.019073 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24227e04-fd14-4515-81b5-686f73bc8643 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:32.019496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24227e04-fd14-4515-81b5-686f73bc8643 None admin] RBAC: Adding query filter params (role.id=5a7214dc069f45969e19c44122b378a4, scope.project.id=0595667fb2e04a3ca4716f669d6698b0, user.id=4b2d638e6bd240ccbe45228352900bdb) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:32.019496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24227e04-fd14-4515-81b5-686f73bc8643 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:32.019496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24227e04-fd14-4515-81b5-686f73bc8643 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:32.020087 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 383/765] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:31 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:54:32.021896 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4098158-1b2e-4075-a09c-94d0d9f7f689 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:32.031059 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 383/766] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:54:31 2022] GET /identity/v3/role_assignments?user.id=4b2d638e6bd240ccbe45228352900bdb&scope.project.id=0595667fb2e04a3ca4716f669d6698b0&role.id=5a7214dc069f45969e19c44122b378a4 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:32.082070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24227e04-fd14-4515-81b5-686f73bc8643 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:32.137797 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:32.161810 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4098158-1b2e-4075-a09c-94d0d9f7f689 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:32.202841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d10e21-67b7-4154-95dd-7c19ddb314e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:32.204226 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d10e21-67b7-4154-95dd-7c19ddb314e2 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:32.204688 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d10e21-67b7-4154-95dd-7c19ddb314e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:32.205163 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d10e21-67b7-4154-95dd-7c19ddb314e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:32.207528 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d10e21-67b7-4154-95dd-7c19ddb314e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:32.208197 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d10e21-67b7-4154-95dd-7c19ddb314e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:32.209870 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d10e21-67b7-4154-95dd-7c19ddb314e2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:32.230820 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 384/767] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:32.233194 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d10e21-67b7-4154-95dd-7c19ddb314e2 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:32.292454 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:32.337349 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24227e04-fd14-4515-81b5-686f73bc8643 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:32.360458 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d10e21-67b7-4154-95dd-7c19ddb314e2 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:32.398298 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42c56ad-2b1a-4624-88d5-bac8809b16b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:32.399618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c56ad-2b1a-4624-88d5-bac8809b16b8 None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:32.400097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c56ad-2b1a-4624-88d5-bac8809b16b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:32.400592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42c56ad-2b1a-4624-88d5-bac8809b16b8 None admin] PATH_INFO: `/v3/projects/0595667fb2e04a3ca4716f669d6698b0/users/225c7b385f12445fab86b3b5606b1da1/roles/17bbe7cc1c3641509de5e9dca8534102` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:32.402997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c56ad-2b1a-4624-88d5-bac8809b16b8 None admin] RBAC: Authorizing `identity:create_grant(project_id=0595667fb2e04a3ca4716f669d6698b0, user_id=225c7b385f12445fab86b3b5606b1da1, role_id=17bbe7cc1c3641509de5e9dca8534102)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:32.404668 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42c56ad-2b1a-4624-88d5-bac8809b16b8 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:32.415960 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:32.426898 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 385/768] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:54:32 2022] PUT /identity/v3/projects/0595667fb2e04a3ca4716f669d6698b0/users/225c7b385f12445fab86b3b5606b1da1/roles/17bbe7cc1c3641509de5e9dca8534102 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:54:32.426898 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdef9b-252e-4ded-971b-9fc43337773a None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:32.426898 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdef9b-252e-4ded-971b-9fc43337773a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:32.426898 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffbdef9b-252e-4ded-971b-9fc43337773a None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:32.426898 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 386/769] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:32.433173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389092a2-0b94-4665-8646-8695f8680e38 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:32.433954 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7f8c2-166b-485b-afc3-9dd360803368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:32.435274 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389092a2-0b94-4665-8646-8695f8680e38 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:32.435555 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389092a2-0b94-4665-8646-8695f8680e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:32.437185 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7f8c2-166b-485b-afc3-9dd360803368 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:32.437545 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7f8c2-166b-485b-afc3-9dd360803368 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:32.437877 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead7f8c2-166b-485b-afc3-9dd360803368 None admin] PATH_INFO: `/v3/services/network` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:32.438566 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7f8c2-166b-485b-afc3-9dd360803368 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:32.440726 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead7f8c2-166b-485b-afc3-9dd360803368 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:32.471070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-ead7f8c2-166b-485b-afc3-9dd360803368 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 23 16:54:32.471070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:32.471070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:54:32.471070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:54:32.471070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:54:32.471070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:54:32.471070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:54:32.471070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:32.471070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 16:54:32.471070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:32.471070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:54:32.471070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:32.471070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:32.471070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:32.471070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:32.471070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:32.471070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:32.472253 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:32.472253 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:32.472253 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:32.472253 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:32.472253 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:32.472253 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:32.472253 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:54:32.472253 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:54:32.472253 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:54:32.472253 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:54:32.472253 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:32.472253 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:32.472253 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:32.472253 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:32.472253 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:32.472253 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:32.473213 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:32.473213 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:32.473213 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:32.473213 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:32.473213 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:32.473213 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:32.473213 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:32.473213 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:32.473213 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:32.473213 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:32.473213 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:54:32.473213 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:32.473213 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 16:54:32.473213 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:32.481228 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 384/770] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:32 2022] GET /identity/v3/services/network => generated 88 bytes in 397 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:32.483202 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-389092a2-0b94-4665-8646-8695f8680e38 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:32.484421 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-389092a2-0b94-4665-8646-8695f8680e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:32.486734 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7f8c2-166b-485b-afc3-9dd360803368 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:32.590033 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 387/771] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:32 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:54:32.624709 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389092a2-0b94-4665-8646-8695f8680e38 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:32.692286 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead7f8c2-166b-485b-afc3-9dd360803368 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:32.724625 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:32.730240 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ebce1-1942-4d69-b900-9f97fe916307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:32.731170 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ebce1-1942-4d69-b900-9f97fe916307 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:32.731669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ebce1-1942-4d69-b900-9f97fe916307 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:32.732177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474ebce1-1942-4d69-b900-9f97fe916307 None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:32.733026 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ebce1-1942-4d69-b900-9f97fe916307 None admin] RBAC: Adding query filter params (name=network) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:32.733473 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ebce1-1942-4d69-b900-9f97fe916307 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:32.735136 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474ebce1-1942-4d69-b900-9f97fe916307 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:32.745620 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 385/772] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 23 16:54:32 2022] GET /identity/v3/services?name=network => generated 124 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:32.747143 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ebce1-1942-4d69-b900-9f97fe916307 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:32.759786 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389092a2-0b94-4665-8646-8695f8680e38 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:32.793103 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:32.812983 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f93fd7-cbf4-4037-9846-1e4beaa49a1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:32.813806 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f93fd7-cbf4-4037-9846-1e4beaa49a1f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:32.814085 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f93fd7-cbf4-4037-9846-1e4beaa49a1f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:32.814356 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f93fd7-cbf4-4037-9846-1e4beaa49a1f None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:32.815090 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f93fd7-cbf4-4037-9846-1e4beaa49a1f None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:32.816553 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f93fd7-cbf4-4037-9846-1e4beaa49a1f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:32.845403 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-36f93fd7-cbf4-4037-9846-1e4beaa49a1f None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 16:54:32.845403 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:32.845403 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:54:32.845403 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:54:32.845403 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:54:32.845403 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:54:32.845403 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:54:32.845403 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:32.845403 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 16:54:32.845403 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:32.845403 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:54:32.845403 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:32.845403 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:32.845403 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:32.845403 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:32.845403 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:32.846585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:32.846585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:32.846585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:32.846585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:32.846585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:32.846585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:32.846585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:32.846585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:54:32.846585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:54:32.846585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:54:32.846585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:54:32.846585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:32.846585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:32.846585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:32.846585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:32.846585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:32.847829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:32.847829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:32.847829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:32.847829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:32.847829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:32.847829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:32.847829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:32.847829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:32.847829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:32.847829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:32.847829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:32.847829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:54:32.847829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:32.847829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 16:54:32.847829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:32.847829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 388/773] 10.222.0.13 () {56 vars in 1129 bytes} [Mon May 23 16:54:32 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:54:32.853790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f93fd7-cbf4-4037-9846-1e4beaa49a1f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:32.869068 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474ebce1-1942-4d69-b900-9f97fe916307 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:32.908134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a3ff8-36ee-4881-9a01-51bfcc2317e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:32.908134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a3ff8-36ee-4881-9a01-51bfcc2317e6 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:32.908134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a3ff8-36ee-4881-9a01-51bfcc2317e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:32.908134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982a3ff8-36ee-4881-9a01-51bfcc2317e6 None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:32.908587 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982a3ff8-36ee-4881-9a01-51bfcc2317e6 None admin] RBAC: Adding query filter params (type=network) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:32.908587 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982a3ff8-36ee-4881-9a01-51bfcc2317e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:32.912406 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982a3ff8-36ee-4881-9a01-51bfcc2317e6 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:32.929634 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 386/774] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 23 16:54:32 2022] GET /identity/v3/services?type=network => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:32.935349 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a3ff8-36ee-4881-9a01-51bfcc2317e6 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:33.046286 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f93fd7-cbf4-4037-9846-1e4beaa49a1f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:33.046850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:33.108144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982a3ff8-36ee-4881-9a01-51bfcc2317e6 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:33.111468 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc9dd9be-25f7-45ec-88b9-8fc1a2f3940f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:33.112155 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9dd9be-25f7-45ec-88b9-8fc1a2f3940f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:33.112507 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9dd9be-25f7-45ec-88b9-8fc1a2f3940f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:33.112919 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9dd9be-25f7-45ec-88b9-8fc1a2f3940f None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:33.113649 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9dd9be-25f7-45ec-88b9-8fc1a2f3940f None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:33.113962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9dd9be-25f7-45ec-88b9-8fc1a2f3940f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:33.115418 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc9dd9be-25f7-45ec-88b9-8fc1a2f3940f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:33.128918 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 389/775] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 23 16:54:32 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 275 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:33.130278 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86f65a-fbb5-47e9-9124-c2d1cdf1d62e None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:33.130564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86f65a-fbb5-47e9-9124-c2d1cdf1d62e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:33.130840 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c86f65a-fbb5-47e9-9124-c2d1cdf1d62e None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:33.133534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 390/776] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:33.135751 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c86f65a-fbb5-47e9-9124-c2d1cdf1d62e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:33.161803 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e4fbce-1ba1-47b8-822b-c96f3f63e6e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:33.162483 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4fbce-1ba1-47b8-822b-c96f3f63e6e1 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:33.162767 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4fbce-1ba1-47b8-822b-c96f3f63e6e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:33.163070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e4fbce-1ba1-47b8-822b-c96f3f63e6e1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:33.163970 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e4fbce-1ba1-47b8-822b-c96f3f63e6e1 None admin] RBAC: Adding query filter params (interface=public, service_id=09024c84597b4d0496d1aad4c217ecd7, region_id=RegionOne) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:33.164292 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e4fbce-1ba1-47b8-822b-c96f3f63e6e1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:33.165786 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e4fbce-1ba1-47b8-822b-c96f3f63e6e1 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:33.178311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 387/777] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 23 16:54:32 2022] GET /identity/v3/endpoints?service_id=09024c84597b4d0496d1aad4c217ecd7&interface=public®ion_id=RegionOne => generated 194 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:33.178311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2474684-47a6-4a95-9a71-7bbf88880370 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:33.178311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2474684-47a6-4a95-9a71-7bbf88880370 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:33.178311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2474684-47a6-4a95-9a71-7bbf88880370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:33.194800 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-c2474684-47a6-4a95-9a71-7bbf88880370 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:33.196134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-c2474684-47a6-4a95-9a71-7bbf88880370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:33.210939 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:33.265001 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c86f65a-fbb5-47e9-9124-c2d1cdf1d62e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:33.291231 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 388/778] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:33 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:54:33.292139 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2474684-47a6-4a95-9a71-7bbf88880370 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:33.303118 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f7712-9e9f-418a-9c17-5de1cacf7a73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:33.304081 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f7712-9e9f-418a-9c17-5de1cacf7a73 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:33.304510 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f7712-9e9f-418a-9c17-5de1cacf7a73 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:33.305073 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556f7712-9e9f-418a-9c17-5de1cacf7a73 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:33.305925 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f7712-9e9f-418a-9c17-5de1cacf7a73 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:33.306381 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f7712-9e9f-418a-9c17-5de1cacf7a73 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:33.308026 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556f7712-9e9f-418a-9c17-5de1cacf7a73 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:33.313923 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 391/779] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 23 16:54:33 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:33.319637 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f7712-9e9f-418a-9c17-5de1cacf7a73 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:33.363999 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:33.412632 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2474684-47a6-4a95-9a71-7bbf88880370 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:33.432827 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556f7712-9e9f-418a-9c17-5de1cacf7a73 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:33.453679 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa801a-eddd-49f5-ad4f-cf7e47646c6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:33.454699 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa801a-eddd-49f5-ad4f-cf7e47646c6b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:33.455169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa801a-eddd-49f5-ad4f-cf7e47646c6b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:33.455573 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa801a-eddd-49f5-ad4f-cf7e47646c6b None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:33.456813 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aa801a-eddd-49f5-ad4f-cf7e47646c6b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:33.459549 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:33.459687 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aa801a-eddd-49f5-ad4f-cf7e47646c6b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:33.472768 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 389/780] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 23 16:54:33 2022] GET /identity/v3/services => generated 772 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:33.472768 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa801a-eddd-49f5-ad4f-cf7e47646c6b None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:33.501580 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9102d1-fb55-4992-a0ad-e66a23b632f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:33.503373 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9102d1-fb55-4992-a0ad-e66a23b632f2 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:33.503663 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9102d1-fb55-4992-a0ad-e66a23b632f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:33.503961 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9102d1-fb55-4992-a0ad-e66a23b632f2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:33.504746 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9102d1-fb55-4992-a0ad-e66a23b632f2 None admin] RBAC: Adding query filter params (interface=public, service_id=e1a1b960b9de41fd85acf1354f690807, region_id=RegionOne) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:33.505058 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9102d1-fb55-4992-a0ad-e66a23b632f2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:33.506477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9102d1-fb55-4992-a0ad-e66a23b632f2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:33.524217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 392/781] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 23 16:54:33 2022] GET /identity/v3/endpoints?service_id=e1a1b960b9de41fd85acf1354f690807&interface=public®ion_id=RegionOne => generated 194 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:33.532254 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9102d1-fb55-4992-a0ad-e66a23b632f2 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:33.667000 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa801a-eddd-49f5-ad4f-cf7e47646c6b None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:33.700799 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:33.712401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9102d1-fb55-4992-a0ad-e66a23b632f2 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:33.721007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4ba23-40eb-46d6-9353-f62e4e57336c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:33.722374 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4ba23-40eb-46d6-9353-f62e4e57336c None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:33.722735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4ba23-40eb-46d6-9353-f62e4e57336c None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:33.723143 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4ba23-40eb-46d6-9353-f62e4e57336c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:33.735399 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4ba23-40eb-46d6-9353-f62e4e57336c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:33.744447 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-28a4ba23-40eb-46d6-9353-f62e4e57336c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:54:33.744447 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:33.744447 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:33.744447 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:33.744447 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:33.744447 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:33.744447 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:33.744447 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:33.744447 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:33.744447 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:33.744447 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:33.744447 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:33.744447 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:33.744447 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:33.744447 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:33.744447 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:33.745367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:33.745367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:33.745367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:33.745367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:33.745367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:33.745367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:33.745367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:33.745367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:33.745367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:33.745367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:33.745367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:33.745367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:33.745367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:33.745367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:33.745367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:33.745367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:33.746178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:33.746178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:33.746178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:33.746178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:33.746178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:33.746178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:33.746178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:33.746178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:33.746178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:33.746178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:33.746178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:54:33.746178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:33.746178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 390/782] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:54:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:33.751678 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4ba23-40eb-46d6-9353-f62e4e57336c None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:33.755396 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b019314-e158-477c-a62e-2fd1394ae6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:33.756800 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b019314-e158-477c-a62e-2fd1394ae6a1 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:33.756800 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b019314-e158-477c-a62e-2fd1394ae6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:33.756800 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b019314-e158-477c-a62e-2fd1394ae6a1 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:33.757524 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b019314-e158-477c-a62e-2fd1394ae6a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:33.759390 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b019314-e158-477c-a62e-2fd1394ae6a1 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:33.765116 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 393/783] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 23 16:54:33 2022] GET /identity/v3/services => generated 772 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:33.876182 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4ba23-40eb-46d6-9353-f62e4e57336c None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:33.933826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312198d3-f5e9-497f-a74a-c0b19a78cde6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:33.934763 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312198d3-f5e9-497f-a74a-c0b19a78cde6 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:33.935297 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312198d3-f5e9-497f-a74a-c0b19a78cde6 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:33.935722 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312198d3-f5e9-497f-a74a-c0b19a78cde6 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:33.936618 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312198d3-f5e9-497f-a74a-c0b19a78cde6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:33.937066 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312198d3-f5e9-497f-a74a-c0b19a78cde6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:33.938616 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312198d3-f5e9-497f-a74a-c0b19a78cde6 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:33.951184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 391/784] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:54:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:33.957097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b019314-e158-477c-a62e-2fd1394ae6a1 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:34.019053 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:34.077261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b019314-e158-477c-a62e-2fd1394ae6a1 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:34.118344 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b59ac-1a6d-49a8-9c93-bb0cffbfef21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:34.119173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b59ac-1a6d-49a8-9c93-bb0cffbfef21 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:34.119450 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b59ac-1a6d-49a8-9c93-bb0cffbfef21 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:34.119718 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6b59ac-1a6d-49a8-9c93-bb0cffbfef21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:34.131063 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b59ac-1a6d-49a8-9c93-bb0cffbfef21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:34.132531 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6b59ac-1a6d-49a8-9c93-bb0cffbfef21 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:34.157239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-cf6b59ac-1a6d-49a8-9c93-bb0cffbfef21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:34.157239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:34.157239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:34.157239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:34.157239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:34.157239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:34.157239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:34.157239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:34.157239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:34.157239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:34.157239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:34.157239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:34.157239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:34.157239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:34.157239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:34.158173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:34.158173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:34.158173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:34.158173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:34.158173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:34.158173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:34.158173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:34.158173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:34.158173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:34.158173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:34.158173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:34.158173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:34.158173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:34.158173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:34.158173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:34.158173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:34.158173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:34.159108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:34.159108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:34.159108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:34.159108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:34.159108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 394/785] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:34.164102 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312198d3-f5e9-497f-a74a-c0b19a78cde6 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:34.219060 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:34.304177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312198d3-f5e9-497f-a74a-c0b19a78cde6 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:34.360570 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6e0f9-ebc1-45a7-8634-8f5bf370fa54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:34.361315 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6e0f9-ebc1-45a7-8634-8f5bf370fa54 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:34.361597 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6e0f9-ebc1-45a7-8634-8f5bf370fa54 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:34.361873 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40a6e0f9-ebc1-45a7-8634-8f5bf370fa54 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:34.362733 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6e0f9-ebc1-45a7-8634-8f5bf370fa54 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:34.363072 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6e0f9-ebc1-45a7-8634-8f5bf370fa54 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:34.364445 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40a6e0f9-ebc1-45a7-8634-8f5bf370fa54 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:34.383319 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 392/786] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:34.392250 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b59ac-1a6d-49a8-9c93-bb0cffbfef21 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:34.558873 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6b59ac-1a6d-49a8-9c93-bb0cffbfef21 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:34.560458 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:34.607516 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e596f7c-4c01-4155-9592-3a802c189a23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:34.608324 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e596f7c-4c01-4155-9592-3a802c189a23 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:34.608600 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e596f7c-4c01-4155-9592-3a802c189a23 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:34.608872 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e596f7c-4c01-4155-9592-3a802c189a23 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:34.618619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e596f7c-4c01-4155-9592-3a802c189a23 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:34.620107 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e596f7c-4c01-4155-9592-3a802c189a23 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:34.637771 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-3e596f7c-4c01-4155-9592-3a802c189a23 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 16:54:34.637771 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:34.637771 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:34.637771 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:34.637771 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:34.637771 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:34.637771 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:34.637771 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:34.637771 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:34.637771 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:34.637771 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:34.637771 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:34.637771 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:34.637771 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:34.637771 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:34.637771 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:34.638787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:34.638787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:34.638787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:34.638787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:34.638787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:34.638787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:34.638787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:34.638787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:34.638787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:34.638787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:34.638787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:34.638787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:34.638787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:34.638787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:34.638787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:34.638787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:34.638787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:34.638787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:34.639785 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:34.639785 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:34.639785 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:34.639785 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:34.639785 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:34.639785 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:34.639785 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:34.639785 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:34.639785 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 16:54:34.639785 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:34.639785 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 395/787] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:54:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:34.647060 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6e0f9-ebc1-45a7-8634-8f5bf370fa54 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:34.709979 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f2178-e0fe-443d-b9fc-ce29d7ad5f7f None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:34.710301 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f2178-e0fe-443d-b9fc-ce29d7ad5f7f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:34.710541 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f2178-e0fe-443d-b9fc-ce29d7ad5f7f None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:34.712149 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 396/788] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:34.724489 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269916d-6b7e-4de0-98d3-624ceb089332 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:34.725210 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269916d-6b7e-4de0-98d3-624ceb089332 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:34.725210 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9269916d-6b7e-4de0-98d3-624ceb089332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:34.751579 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-9269916d-6b7e-4de0-98d3-624ceb089332 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:34.755328 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9269916d-6b7e-4de0-98d3-624ceb089332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:34.792404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:34.796752 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40a6e0f9-ebc1-45a7-8634-8f5bf370fa54 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:34.864050 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1da2a-c4ee-42f8-850a-daf9e3380027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:34.867680 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1da2a-c4ee-42f8-850a-daf9e3380027 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:34.868192 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1da2a-c4ee-42f8-850a-daf9e3380027 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:34.868679 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa1da2a-c4ee-42f8-850a-daf9e3380027 None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:34.869720 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1da2a-c4ee-42f8-850a-daf9e3380027 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:34.870432 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1da2a-c4ee-42f8-850a-daf9e3380027 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:34.873053 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa1da2a-c4ee-42f8-850a-daf9e3380027 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:34.907116 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 397/789] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:34 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 16:54:34.912518 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 393/790] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 23 16:54:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:34.917688 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269916d-6b7e-4de0-98d3-624ceb089332 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:34.928260 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1da2a-c4ee-42f8-850a-daf9e3380027 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:35.039537 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa1da2a-c4ee-42f8-850a-daf9e3380027 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:35.062716 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:35.077578 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9269916d-6b7e-4de0-98d3-624ceb089332 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:35.080955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6416d-6571-4291-8c66-0d74141bdc28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:35.082014 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6416d-6571-4291-8c66-0d74141bdc28 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:35.082497 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6416d-6571-4291-8c66-0d74141bdc28 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:35.083105 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b6416d-6571-4291-8c66-0d74141bdc28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:35.094298 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9b6416d-6571-4291-8c66-0d74141bdc28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:35.102807 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-f9b6416d-6571-4291-8c66-0d74141bdc28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:35.102807 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:35.102807 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:35.102807 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:35.102807 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:35.102807 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:35.102807 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:35.102807 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:35.102807 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:35.102807 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:35.102807 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:35.102807 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:35.102807 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:35.102807 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:35.102807 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:35.102807 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:35.103857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:35.103857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:35.103857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:35.103857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:35.103857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:35.103857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:35.103857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:35.103857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:35.103857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:35.103857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:35.103857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:35.103857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:35.103857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:35.103857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:35.103857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:35.103857 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:35.104704 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:35.104704 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:35.104704 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:35.104704 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:35.104704 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:35.104704 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:35.104704 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:35.104704 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:35.104704 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:35.104704 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:35.104704 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:35.104704 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:35.107719 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 394/791] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 23 16:54:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:54:35.114198 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6416d-6571-4291-8c66-0d74141bdc28 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:35.137738 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:35.146945 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bac5fe-1c6a-4f06-b389-c588bbcc60df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:35.147569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bac5fe-1c6a-4f06-b389-c588bbcc60df None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:35.147847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bac5fe-1c6a-4f06-b389-c588bbcc60df None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:35.148115 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bac5fe-1c6a-4f06-b389-c588bbcc60df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:35.172062 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bac5fe-1c6a-4f06-b389-c588bbcc60df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:35.173545 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bac5fe-1c6a-4f06-b389-c588bbcc60df None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:35.196596 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-c7bac5fe-1c6a-4f06-b389-c588bbcc60df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:35.196596 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:35.196596 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:35.196596 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:35.196596 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:35.196596 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:35.196596 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:35.196596 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:35.196596 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:35.196596 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:35.196596 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:35.196596 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:35.196596 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:35.196596 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:35.196596 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:35.197562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:35.197562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:35.197562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:35.197562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:35.197562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:35.197562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:35.197562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:35.197562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:35.197562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:35.197562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:35.197562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:35.197562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:35.197562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:35.197562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:35.197562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:35.197562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:35.197562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:35.198448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:35.198448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:35.198448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:35.198448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:35.198448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 398/792] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:35.203598 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bac5fe-1c6a-4f06-b389-c588bbcc60df None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:35.231647 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b6416d-6571-4291-8c66-0d74141bdc28 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:35.269985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4755238-1bc2-4033-84a9-4fe79d4daa48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:35.271101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4755238-1bc2-4033-84a9-4fe79d4daa48 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:35.271589 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4755238-1bc2-4033-84a9-4fe79d4daa48 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:35.272065 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4755238-1bc2-4033-84a9-4fe79d4daa48 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:35.273179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4755238-1bc2-4033-84a9-4fe79d4daa48 None admin] RBAC: Adding query filter params (name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:35.273688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4755238-1bc2-4033-84a9-4fe79d4daa48 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:35.275317 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4755238-1bc2-4033-84a9-4fe79d4daa48 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:35.285147 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 395/793] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 23 16:54:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:35.291519 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57dd882-3807-4711-83fb-a37c498bcc26 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:35.292065 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57dd882-3807-4711-83fb-a37c498bcc26 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:35.292542 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57dd882-3807-4711-83fb-a37c498bcc26 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:35.294513 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 396/794] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:35.295607 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57dd882-3807-4711-83fb-a37c498bcc26 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:35.361894 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bac5fe-1c6a-4f06-b389-c588bbcc60df None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:35.391916 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:35.399383 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8d2082-9944-4388-a56e-db4ea5101d05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:35.400055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d2082-9944-4388-a56e-db4ea5101d05 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:35.400337 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d2082-9944-4388-a56e-db4ea5101d05 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:35.400609 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8d2082-9944-4388-a56e-db4ea5101d05 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:35.401336 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8d2082-9944-4388-a56e-db4ea5101d05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:35.401709 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8d2082-9944-4388-a56e-db4ea5101d05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:35.403677 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8d2082-9944-4388-a56e-db4ea5101d05 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:35.415979 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 399/795] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:35.417340 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52502937-4f02-443b-ade0-d374ba6578ca None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:35.417633 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52502937-4f02-443b-ade0-d374ba6578ca None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:35.417909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52502937-4f02-443b-ade0-d374ba6578ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:35.420527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57dd882-3807-4711-83fb-a37c498bcc26 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:35.425491 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:35.437046 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-52502937-4f02-443b-ade0-d374ba6578ca None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:35.438336 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-52502937-4f02-443b-ade0-d374ba6578ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:35.471492 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428955c1-faf5-4f95-9b86-8506f92d407e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:35.476221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428955c1-faf5-4f95-9b86-8506f92d407e None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:35.476759 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428955c1-faf5-4f95-9b86-8506f92d407e None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:35.477239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-428955c1-faf5-4f95-9b86-8506f92d407e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:35.490080 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428955c1-faf5-4f95-9b86-8506f92d407e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:35.493402 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-428955c1-faf5-4f95-9b86-8506f92d407e None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:35.514304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-428955c1-faf5-4f95-9b86-8506f92d407e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:35.514304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:35.514304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:35.514304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:35.514304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:35.514304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:35.514304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:35.514304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:35.514304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:35.514304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:35.514304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:35.514304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:35.514304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:35.514304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:35.514304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:35.515249 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:35.515249 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:35.515249 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:35.515249 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:35.515249 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:35.515249 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:35.515249 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:35.515249 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:35.515249 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:35.515249 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:35.515249 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:35.515249 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:35.515249 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:35.515249 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:35.515249 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:35.515249 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:35.515249 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:35.516132 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:35.516132 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:35.516132 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:35.516132 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:35.518411 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 397/796] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:35.519648 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428955c1-faf5-4f95-9b86-8506f92d407e None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:35.534962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 400/797] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:35 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:54:35.534962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52502937-4f02-443b-ade0-d374ba6578ca None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:35.651209 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52502937-4f02-443b-ade0-d374ba6578ca None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:35.656929 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-428955c1-faf5-4f95-9b86-8506f92d407e None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:35.690680 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1ec12-876c-4c32-8a55-a5e9d678c116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:35.691876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1ec12-876c-4c32-8a55-a5e9d678c116 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:35.692356 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1ec12-876c-4c32-8a55-a5e9d678c116 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:35.692821 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e1ec12-876c-4c32-8a55-a5e9d678c116 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:35.693742 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1ec12-876c-4c32-8a55-a5e9d678c116 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:35.694247 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1ec12-876c-4c32-8a55-a5e9d678c116 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:35.695892 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2e1ec12-876c-4c32-8a55-a5e9d678c116 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:35.696413 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:35.699069 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:35.705894 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8a1be-31ee-434c-9844-eb6146af2ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:35.707277 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8a1be-31ee-434c-9844-eb6146af2ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:35.708292 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8a1be-31ee-434c-9844-eb6146af2ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:35.708747 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8a1be-31ee-434c-9844-eb6146af2ed4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:35.717861 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 401/798] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:35.717861 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1ec12-876c-4c32-8a55-a5e9d678c116 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:35.723227 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8a1be-31ee-434c-9844-eb6146af2ed4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:35.724816 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f8a1be-31ee-434c-9844-eb6146af2ed4 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:35.740349 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-28f8a1be-31ee-434c-9844-eb6146af2ed4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:35.740349 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:35.740349 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:35.740349 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:35.740349 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:35.740349 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:35.740349 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:35.740349 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:35.740349 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:35.740349 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:35.740349 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:35.740349 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:35.740349 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:35.740349 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:35.740349 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:35.741499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:35.741499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:35.741499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:35.741499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:35.741499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:35.741499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:35.741499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:35.741499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:35.741499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:35.741499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:35.741499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:35.741499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:35.741499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:35.741499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:35.741499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:35.741499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:35.741499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:35.742610 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:35.742610 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:35.742610 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:35.742610 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:35.742610 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:35.742610 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:35.742610 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:35.742610 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:35.744482 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 398/799] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:35.745474 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8a1be-31ee-434c-9844-eb6146af2ed4 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:35.851094 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8a1be-31ee-434c-9844-eb6146af2ed4 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:35.858173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2e1ec12-876c-4c32-8a55-a5e9d678c116 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:35.889338 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9335c-66d3-4c03-b076-5461ab72022a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:35.891135 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9335c-66d3-4c03-b076-5461ab72022a None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:35.891573 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9335c-66d3-4c03-b076-5461ab72022a None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:35.892000 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f9335c-66d3-4c03-b076-5461ab72022a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:35.902001 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bc664-0b9b-429b-b989-6385242c5d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:35.902337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9335c-66d3-4c03-b076-5461ab72022a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:35.903017 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bc664-0b9b-429b-b989-6385242c5d75 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:35.903394 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bc664-0b9b-429b-b989-6385242c5d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:35.903764 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b75bc664-0b9b-429b-b989-6385242c5d75 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:35.904538 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f9335c-66d3-4c03-b076-5461ab72022a None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:35.915041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b75bc664-0b9b-429b-b989-6385242c5d75 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:35.923047 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-b75bc664-0b9b-429b-b989-6385242c5d75 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:54:35.923047 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:35.923047 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:35.923047 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:35.923047 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:35.923047 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:35.923047 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:35.923047 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:35.923047 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:35.923047 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:35.923047 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:35.923047 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:35.923047 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:35.923047 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:35.923047 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:35.923047 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:35.924175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:35.924175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:35.924175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:35.924175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:35.924175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:35.924175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:35.924175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:35.924175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:35.924175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:35.924175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:35.924175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:35.924175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:35.924175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:35.924175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:35.924175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:35.924175 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:35.925250 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:35.925250 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:35.925250 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:35.925250 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:35.925250 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:35.925250 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:35.925250 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:35.925250 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:35.925250 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:35.925250 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:35.925250 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:54:35.925250 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:35.926295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-a2f9335c-66d3-4c03-b076-5461ab72022a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 16:54:35.926295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:35.926295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:35.926295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:35.926295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:35.926295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:35.926295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:35.926295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:35.926295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:35.926295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:35.926295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:35.926295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:35.926295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:35.926295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:35.926295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:35.926295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:35.927404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:35.927404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:35.927404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:35.927404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:35.927404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:35.927404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:35.927404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:35.927404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:35.927404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:35.927404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:35.927404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:35.927404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:35.927404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:35.927404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:35.927404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:35.927404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:35.927404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:35.927404 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:35.928551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:35.928551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:35.928551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:35.928551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:35.928551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:35.928551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:35.928551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:35.928551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:35.928551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 16:54:35.928551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:35.935243 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 402/801] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:54:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:35.935491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 399/801] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 23 16:54:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:35.935892 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bc664-0b9b-429b-b989-6385242c5d75 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:35.937035 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9335c-66d3-4c03-b076-5461ab72022a None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:35.997288 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:35.998961 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:36.049662 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b75bc664-0b9b-429b-b989-6385242c5d75 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:36.057346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f9335c-66d3-4c03-b076-5461ab72022a None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:36.084895 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90482eb7-2120-40e1-9ae5-75c4a8ab82da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:36.085568 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90482eb7-2120-40e1-9ae5-75c4a8ab82da None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:36.085850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90482eb7-2120-40e1-9ae5-75c4a8ab82da None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:36.086125 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90482eb7-2120-40e1-9ae5-75c4a8ab82da None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:36.086935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90482eb7-2120-40e1-9ae5-75c4a8ab82da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:36.087320 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90482eb7-2120-40e1-9ae5-75c4a8ab82da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:36.088923 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90482eb7-2120-40e1-9ae5-75c4a8ab82da None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:36.096826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f01c77-7169-4b0e-ab1d-7f4e1261132e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:36.097739 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f01c77-7169-4b0e-ab1d-7f4e1261132e None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:36.098325 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f01c77-7169-4b0e-ab1d-7f4e1261132e None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:36.098789 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f01c77-7169-4b0e-ab1d-7f4e1261132e None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:36.099825 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f01c77-7169-4b0e-ab1d-7f4e1261132e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:36.100316 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f01c77-7169-4b0e-ab1d-7f4e1261132e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:36.101975 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f01c77-7169-4b0e-ab1d-7f4e1261132e None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:36.106452 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 403/802] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:36.107505 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90482eb7-2120-40e1-9ae5-75c4a8ab82da None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:36.128303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 400/803] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 23 16:54:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:36.129690 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f01c77-7169-4b0e-ab1d-7f4e1261132e None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:36.214757 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90482eb7-2120-40e1-9ae5-75c4a8ab82da None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:36.229516 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:36.243789 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f01c77-7169-4b0e-ab1d-7f4e1261132e None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:36.251395 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293352ad-e0f3-4d6a-a802-da17b8afc2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:36.252079 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293352ad-e0f3-4d6a-a802-da17b8afc2d6 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:36.252361 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293352ad-e0f3-4d6a-a802-da17b8afc2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:36.252635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-293352ad-e0f3-4d6a-a802-da17b8afc2d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:36.253811 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293352ad-e0f3-4d6a-a802-da17b8afc2d6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:36.253986 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293352ad-e0f3-4d6a-a802-da17b8afc2d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:36.255606 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-293352ad-e0f3-4d6a-a802-da17b8afc2d6 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:36.256038 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:36.269519 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 404/804] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:54:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:36.269519 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293352ad-e0f3-4d6a-a802-da17b8afc2d6 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:36.281011 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5005df50-e68e-4e6b-bec5-cb25257ab89d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:36.282071 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005df50-e68e-4e6b-bec5-cb25257ab89d None admin] REQUEST_METHOD: `PUT` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:36.282351 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005df50-e68e-4e6b-bec5-cb25257ab89d None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:36.282670 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5005df50-e68e-4e6b-bec5-cb25257ab89d None admin] PATH_INFO: `/v3/projects/0595667fb2e04a3ca4716f669d6698b0/users/6546217667054abb8eb07076995c1b04/roles/17bbe7cc1c3641509de5e9dca8534102` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:36.284917 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005df50-e68e-4e6b-bec5-cb25257ab89d None admin] RBAC: Authorizing `identity:create_grant(project_id=0595667fb2e04a3ca4716f669d6698b0, user_id=6546217667054abb8eb07076995c1b04, role_id=17bbe7cc1c3641509de5e9dca8534102)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:36.286485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5005df50-e68e-4e6b-bec5-cb25257ab89d None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:36.301471 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 401/805] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:54:36 2022] PUT /identity/v3/projects/0595667fb2e04a3ca4716f669d6698b0/users/6546217667054abb8eb07076995c1b04/roles/17bbe7cc1c3641509de5e9dca8534102 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:54:36.302838 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ceb2a-6aef-4b7c-b44e-80aa283306e9 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:36.303153 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ceb2a-6aef-4b7c-b44e-80aa283306e9 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:36.303432 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95ceb2a-6aef-4b7c-b44e-80aa283306e9 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:36.306996 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 402/806] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:36.306996 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ceb2a-6aef-4b7c-b44e-80aa283306e9 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:36.389665 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-293352ad-e0f3-4d6a-a802-da17b8afc2d6 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:36.428828 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff57d87-0850-4743-92ab-60c514238bf2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:36.429635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff57d87-0850-4743-92ab-60c514238bf2 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:36.429909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff57d87-0850-4743-92ab-60c514238bf2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:36.430176 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff57d87-0850-4743-92ab-60c514238bf2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:36.432163 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95ceb2a-6aef-4b7c-b44e-80aa283306e9 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:36.443447 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff57d87-0850-4743-92ab-60c514238bf2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:36.445054 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff57d87-0850-4743-92ab-60c514238bf2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:36.465098 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-eff57d87-0850-4743-92ab-60c514238bf2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:36.465098 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:36.465098 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:36.465098 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:36.465098 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:36.465098 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:36.465098 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:36.465098 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:36.465098 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:36.465098 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:36.465098 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:36.465098 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:36.465098 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:36.465098 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:36.465098 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:36.466099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:36.466099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:36.466099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:36.466099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:36.466099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:36.466099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:36.466099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:36.466099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:36.466099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:36.466099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:36.466099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:36.466099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:36.466099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:36.466099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:36.466099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:36.466099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:36.466099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:36.472204 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:36.472204 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:36.472204 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:36.472204 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:36.472204 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 405/807] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:36.472204 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ce0f2-8012-4cc3-a64d-6ef69ee34f90 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:36.472204 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ce0f2-8012-4cc3-a64d-6ef69ee34f90 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:36.472204 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5ce0f2-8012-4cc3-a64d-6ef69ee34f90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:36.483343 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05574777-16b2-4fc8-ae52-be7c73302314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:36.484209 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05574777-16b2-4fc8-ae52-be7c73302314 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:36.484445 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05574777-16b2-4fc8-ae52-be7c73302314 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:36.485193 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05574777-16b2-4fc8-ae52-be7c73302314 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:36.490314 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-0e5ce0f2-8012-4cc3-a64d-6ef69ee34f90 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:36.491177 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0e5ce0f2-8012-4cc3-a64d-6ef69ee34f90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:36.500534 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05574777-16b2-4fc8-ae52-be7c73302314 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:36.502049 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05574777-16b2-4fc8-ae52-be7c73302314 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:36.518834 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:36.523208 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-05574777-16b2-4fc8-ae52-be7c73302314 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:36.523208 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:36.523208 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:36.523208 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:36.523208 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:36.523208 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:36.523208 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:36.523208 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:36.523208 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:36.523208 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:36.523208 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:36.523208 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:36.523208 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:36.523208 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:36.523208 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:36.524264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:36.524264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:36.524264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:36.524264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:36.524264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:36.524264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:36.524264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:36.524264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:36.524264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:36.524264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:36.524264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:36.524264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:36.524264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:36.524264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:36.524264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:36.524264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:36.524264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:36.525336 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:36.525336 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:36.525336 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:36.525336 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:36.526450 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 403/808] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:36.527911 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561064bd-6dda-4ecb-83f0-de28748a6f26 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:36.528195 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561064bd-6dda-4ecb-83f0-de28748a6f26 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:36.528471 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561064bd-6dda-4ecb-83f0-de28748a6f26 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:36.529842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 404/809] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:36.531445 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561064bd-6dda-4ecb-83f0-de28748a6f26 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:36.564585 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:36.576998 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 406/810] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:54:36.580396 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ce0f2-8012-4cc3-a64d-6ef69ee34f90 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:36.654529 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561064bd-6dda-4ecb-83f0-de28748a6f26 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:36.693312 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c66fde6-05d9-4b2b-acec-bfe7be510105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:36.693895 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66fde6-05d9-4b2b-acec-bfe7be510105 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:36.694171 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66fde6-05d9-4b2b-acec-bfe7be510105 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:36.694442 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c66fde6-05d9-4b2b-acec-bfe7be510105 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:36.695194 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c66fde6-05d9-4b2b-acec-bfe7be510105 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:36.695553 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c66fde6-05d9-4b2b-acec-bfe7be510105 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:36.696924 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c66fde6-05d9-4b2b-acec-bfe7be510105 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:36.713981 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 405/811] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:36.714184 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5ce0f2-8012-4cc3-a64d-6ef69ee34f90 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:36.714283 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50327f2e-9956-4337-a8e6-5102247e5cd7 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:36.714283 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50327f2e-9956-4337-a8e6-5102247e5cd7 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:36.714283 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50327f2e-9956-4337-a8e6-5102247e5cd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:36.728520 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-50327f2e-9956-4337-a8e6-5102247e5cd7 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:36.729615 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-50327f2e-9956-4337-a8e6-5102247e5cd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:36.749997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916fadc-05b5-4d54-b30b-f2cc394ff6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:36.750667 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916fadc-05b5-4d54-b30b-f2cc394ff6e8 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:36.750968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916fadc-05b5-4d54-b30b-f2cc394ff6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:36.751242 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3916fadc-05b5-4d54-b30b-f2cc394ff6e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:36.751959 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916fadc-05b5-4d54-b30b-f2cc394ff6e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:36.752262 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916fadc-05b5-4d54-b30b-f2cc394ff6e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:36.753686 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3916fadc-05b5-4d54-b30b-f2cc394ff6e8 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:36.771267 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 407/812] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:36.771267 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916fadc-05b5-4d54-b30b-f2cc394ff6e8 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:36.794634 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:36.808358 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 406/813] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:36 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:54:36.809342 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50327f2e-9956-4337-a8e6-5102247e5cd7 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:36.823660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:36.901427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3916fadc-05b5-4d54-b30b-f2cc394ff6e8 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:36.933377 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50327f2e-9956-4337-a8e6-5102247e5cd7 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:36.939086 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5a1ba-ff86-4077-b402-e9df1abbe2ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:36.940165 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5a1ba-ff86-4077-b402-e9df1abbe2ad None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:36.940798 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5a1ba-ff86-4077-b402-e9df1abbe2ad None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:36.941296 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc5a1ba-ff86-4077-b402-e9df1abbe2ad None admin] PATH_INFO: `/v3/services/network` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:36.942168 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5a1ba-ff86-4077-b402-e9df1abbe2ad None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:36.943603 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc5a1ba-ff86-4077-b402-e9df1abbe2ad None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:36.957413 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-3cc5a1ba-ff86-4077-b402-e9df1abbe2ad None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 23 16:54:36.957413 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:36.957413 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:54:36.957413 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:54:36.957413 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:54:36.957413 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:54:36.957413 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:54:36.957413 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:36.957413 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 16:54:36.957413 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:36.957413 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:54:36.957413 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:36.957413 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:36.957413 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:36.957413 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:36.957413 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:36.957413 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:36.958659 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:36.958659 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:36.958659 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:36.958659 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:36.958659 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:36.958659 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:36.958659 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:54:36.958659 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:54:36.958659 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:54:36.958659 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:54:36.958659 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:36.958659 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:36.958659 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:36.958659 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:36.958659 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:36.958659 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:36.959537 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:36.959537 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:36.959537 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:36.959537 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:36.959537 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:36.959537 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:36.959537 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:36.959537 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:36.959537 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:36.959537 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:36.959537 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:54:36.959537 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:36.959537 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 16:54:36.959537 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:36.959537 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 408/814] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:36 2022] GET /identity/v3/services/network => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:36.960739 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5a1ba-ff86-4077-b402-e9df1abbe2ad None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:36.981915 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe4b74-1f21-4cc9-ab4c-991195b471b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:36.983306 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe4b74-1f21-4cc9-ab4c-991195b471b2 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:36.984026 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe4b74-1f21-4cc9-ab4c-991195b471b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:36.984514 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbe4b74-1f21-4cc9-ab4c-991195b471b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:36.995765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe4b74-1f21-4cc9-ab4c-991195b471b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:36.998158 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbe4b74-1f21-4cc9-ab4c-991195b471b2 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:37.014140 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-6fbe4b74-1f21-4cc9-ab4c-991195b471b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:37.014140 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:37.014140 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:37.014140 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:37.014140 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:37.014140 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:37.014140 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:37.014140 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:37.014140 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:37.014140 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:37.014140 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:37.014140 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:37.014140 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:37.014140 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:37.014140 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:37.015293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:37.015293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:37.015293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:37.015293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:37.015293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:37.015293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:37.015293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:37.015293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:37.015293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:37.015293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:37.015293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:37.015293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:37.015293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:37.015293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:37.015293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:37.015293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:37.015293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:37.016350 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:37.016350 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:37.016350 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:37.016350 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:37.016350 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:37.016350 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:37.016350 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:37.016350 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:37.017413 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 407/815] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:37.018323 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe4b74-1f21-4cc9-ab4c-991195b471b2 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:37.072256 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:37.095246 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:37.098601 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc5a1ba-ff86-4077-b402-e9df1abbe2ad None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:37.130702 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbe4b74-1f21-4cc9-ab4c-991195b471b2 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:37.138299 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622d5ea-5373-4271-ba4c-5ee4195d5fb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:37.139239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622d5ea-5373-4271-ba4c-5ee4195d5fb3 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:37.139520 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622d5ea-5373-4271-ba4c-5ee4195d5fb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:37.139790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b622d5ea-5373-4271-ba4c-5ee4195d5fb3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:37.148638 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b622d5ea-5373-4271-ba4c-5ee4195d5fb3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:37.150108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b622d5ea-5373-4271-ba4c-5ee4195d5fb3 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:37.167217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-b622d5ea-5373-4271-ba4c-5ee4195d5fb3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 16:54:37.167217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:37.167217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:37.167217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:37.167217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:37.167217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:37.167217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:37.167217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:37.167217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:37.167217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:37.167217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:37.167217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:37.167217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:37.167217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:37.167217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:37.167217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:37.168428 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:37.168428 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:37.168428 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:37.168428 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:37.168428 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:37.168428 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:37.168428 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:37.168428 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:37.168428 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:37.168428 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:37.168428 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:37.168428 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:37.168428 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:37.168428 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:37.168428 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:37.168428 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:37.168428 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:37.168428 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:37.169392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:37.169392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:37.169392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:37.169392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:37.169392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:37.169392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:37.169392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:37.169392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:37.169392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 16:54:37.169392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:37.169392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 409/816] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 23 16:54:36 2022] GET /identity/v3/users/nova => generated 82 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:37.169983 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622d5ea-5373-4271-ba4c-5ee4195d5fb3 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:37.174603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf500a1-b377-4d60-bfe4-e647e46e2cef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:37.175539 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf500a1-b377-4d60-bfe4-e647e46e2cef None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:37.175818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf500a1-b377-4d60-bfe4-e647e46e2cef None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:37.176080 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf500a1-b377-4d60-bfe4-e647e46e2cef None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:37.176743 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf500a1-b377-4d60-bfe4-e647e46e2cef None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:37.178123 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf500a1-b377-4d60-bfe4-e647e46e2cef None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:37.191636 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-cbf500a1-b377-4d60-bfe4-e647e46e2cef None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 16:54:37.191636 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:37.191636 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:54:37.191636 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:54:37.191636 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:54:37.191636 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:54:37.191636 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:54:37.191636 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:37.191636 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 16:54:37.191636 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:37.191636 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:54:37.191636 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:37.191636 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:37.191636 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:37.191636 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:37.191636 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:37.193735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:37.193735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:37.193735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:37.193735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:37.193735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:37.193735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:37.193735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:37.193735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:54:37.193735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:54:37.193735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:54:37.193735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:54:37.193735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:37.193735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:37.193735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:37.193735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:37.193735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:37.195686 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:37.195686 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:37.195686 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:37.195686 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:37.195686 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:37.195686 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:37.195686 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:37.195686 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:37.195686 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:37.195686 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:37.195686 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:37.195686 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:54:37.195686 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:37.195686 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 16:54:37.195686 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:37.198084 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 408/817] 10.222.0.13 () {56 vars in 1129 bytes} [Mon May 23 16:54:37 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:37.199014 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf500a1-b377-4d60-bfe4-e647e46e2cef None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:37.314968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b622d5ea-5373-4271-ba4c-5ee4195d5fb3 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:37.316924 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf500a1-b377-4d60-bfe4-e647e46e2cef None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:37.353993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc8f463-3a16-4e18-ad04-732594fe8b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:37.354662 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8f463-3a16-4e18-ad04-732594fe8b91 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:37.354967 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8f463-3a16-4e18-ad04-732594fe8b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:37.355443 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc8f463-3a16-4e18-ad04-732594fe8b91 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:37.356166 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc8f463-3a16-4e18-ad04-732594fe8b91 None admin] RBAC: Adding query filter params (name=network) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:37.356481 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc8f463-3a16-4e18-ad04-732594fe8b91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:37.357869 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc8f463-3a16-4e18-ad04-732594fe8b91 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:37.360658 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a624fb-c215-4ae2-aaee-0abd3f0852c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:37.361391 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a624fb-c215-4ae2-aaee-0abd3f0852c9 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:37.361675 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a624fb-c215-4ae2-aaee-0abd3f0852c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:37.362130 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a624fb-c215-4ae2-aaee-0abd3f0852c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:37.368360 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 410/818] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 23 16:54:37 2022] GET /identity/v3/services?name=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:37.368530 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a624fb-c215-4ae2-aaee-0abd3f0852c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:37.368530 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a624fb-c215-4ae2-aaee-0abd3f0852c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:37.368682 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc8f463-3a16-4e18-ad04-732594fe8b91 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:37.368755 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a624fb-c215-4ae2-aaee-0abd3f0852c9 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:37.372107 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:37.378014 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:37.385218 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 409/819] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:37.390316 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a624fb-c215-4ae2-aaee-0abd3f0852c9 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:37.494414 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc8f463-3a16-4e18-ad04-732594fe8b91 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:37.503378 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a624fb-c215-4ae2-aaee-0abd3f0852c9 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:37.531958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c6b58-f6dd-439b-ad9f-97e367e66fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:37.532646 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c6b58-f6dd-439b-ad9f-97e367e66fe5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:37.532935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c6b58-f6dd-439b-ad9f-97e367e66fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:37.533207 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859c6b58-f6dd-439b-ad9f-97e367e66fe5 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:37.534072 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c6b58-f6dd-439b-ad9f-97e367e66fe5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:37.534779 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c6b58-f6dd-439b-ad9f-97e367e66fe5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:37.537699 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859c6b58-f6dd-439b-ad9f-97e367e66fe5 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:37.546470 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c947a2-e421-4e79-9e7e-5e222c259b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:37.547479 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c947a2-e421-4e79-9e7e-5e222c259b03 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:37.547747 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c947a2-e421-4e79-9e7e-5e222c259b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:37.548032 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c947a2-e421-4e79-9e7e-5e222c259b03 None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:37.549791 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c947a2-e421-4e79-9e7e-5e222c259b03 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:37.549791 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c947a2-e421-4e79-9e7e-5e222c259b03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:37.550707 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c947a2-e421-4e79-9e7e-5e222c259b03 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:37.556549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 410/820] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 23 16:54:37 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:37.557397 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c947a2-e421-4e79-9e7e-5e222c259b03 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:37.564823 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 411/821] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 23 16:54:37 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:37.568930 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c6b58-f6dd-439b-ad9f-97e367e66fe5 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:37.627421 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:37.670035 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:37.684391 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c947a2-e421-4e79-9e7e-5e222c259b03 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:37.708691 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859c6b58-f6dd-439b-ad9f-97e367e66fe5 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:37.729123 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa23287-87e3-4da9-9d8a-a4dc7585096a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:37.729796 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa23287-87e3-4da9-9d8a-a4dc7585096a None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:37.730081 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa23287-87e3-4da9-9d8a-a4dc7585096a None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:37.730355 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa23287-87e3-4da9-9d8a-a4dc7585096a None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:37.731102 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa23287-87e3-4da9-9d8a-a4dc7585096a None admin] RBAC: Adding query filter params (type=network) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:37.731401 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa23287-87e3-4da9-9d8a-a4dc7585096a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:37.732851 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa23287-87e3-4da9-9d8a-a4dc7585096a None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:37.738872 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 411/822] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 23 16:54:37 2022] GET /identity/v3/services?type=network => generated 350 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:37.739702 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa23287-87e3-4da9-9d8a-a4dc7585096a None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:37.754356 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4514166-fb14-4bf0-b00c-31db753feaab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:37.755289 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4514166-fb14-4bf0-b00c-31db753feaab None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:37.755622 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4514166-fb14-4bf0-b00c-31db753feaab None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:37.755902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4514166-fb14-4bf0-b00c-31db753feaab None admin] PATH_INFO: `/v3/projects/0595667fb2e04a3ca4716f669d6698b0/users/4b2d638e6bd240ccbe45228352900bdb/roles/5a7214dc069f45969e19c44122b378a4` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:37.772281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4514166-fb14-4bf0-b00c-31db753feaab None admin] RBAC: Authorizing `identity:create_grant(project_id=0595667fb2e04a3ca4716f669d6698b0, user_id=4b2d638e6bd240ccbe45228352900bdb, role_id=5a7214dc069f45969e19c44122b378a4)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:37.773778 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4514166-fb14-4bf0-b00c-31db753feaab None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:37.797661 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 412/823] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:54:37 2022] PUT /identity/v3/projects/0595667fb2e04a3ca4716f669d6698b0/users/4b2d638e6bd240ccbe45228352900bdb/roles/5a7214dc069f45969e19c44122b378a4 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:54:37.797661 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4514166-fb14-4bf0-b00c-31db753feaab None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:37.872693 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa23287-87e3-4da9-9d8a-a4dc7585096a None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:37.905891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:37.917357 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4514166-fb14-4bf0-b00c-31db753feaab None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:37.923482 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a0833a-8864-421f-8929-519db6a40500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:37.924317 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a0833a-8864-421f-8929-519db6a40500 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:37.924708 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a0833a-8864-421f-8929-519db6a40500 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:37.925104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a0833a-8864-421f-8929-519db6a40500 None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:37.925909 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a0833a-8864-421f-8929-519db6a40500 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:37.926293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a0833a-8864-421f-8929-519db6a40500 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:37.928204 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a0833a-8864-421f-8929-519db6a40500 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:37.941719 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 412/824] 10.222.0.13 () {56 vars in 1124 bytes} [Mon May 23 16:54:37 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:37.941719 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a0833a-8864-421f-8929-519db6a40500 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:37.945800 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:37.973802 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9de11d-49ee-4bb1-b45b-cd84608dd6fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:37.974467 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9de11d-49ee-4bb1-b45b-cd84608dd6fa None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:37.974790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9de11d-49ee-4bb1-b45b-cd84608dd6fa None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:37.975094 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9de11d-49ee-4bb1-b45b-cd84608dd6fa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:38.056317 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9de11d-49ee-4bb1-b45b-cd84608dd6fa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:38.057781 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9de11d-49ee-4bb1-b45b-cd84608dd6fa None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:38.086715 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a0833a-8864-421f-8929-519db6a40500 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:38.137268 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ed9a1-b591-438c-acba-8d43ef9e8a27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:38.138873 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ed9a1-b591-438c-acba-8d43ef9e8a27 None admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:38.139356 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ed9a1-b591-438c-acba-8d43ef9e8a27 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:38.139929 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305ed9a1-b591-438c-acba-8d43ef9e8a27 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:38.141270 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ed9a1-b591-438c-acba-8d43ef9e8a27 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:38.143725 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305ed9a1-b591-438c-acba-8d43ef9e8a27 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:38.174503 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 413/825] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:54:37 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 16:54:38.174503 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9de11d-49ee-4bb1-b45b-cd84608dd6fa None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:38.180254 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 413/826] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 23 16:54:37 2022] POST /identity/v3/endpoints => generated 327 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:54:38.185771 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ed9a1-b591-438c-acba-8d43ef9e8a27 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:38.213414 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:38.250390 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:38.340594 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305ed9a1-b591-438c-acba-8d43ef9e8a27 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:38.401483 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9de11d-49ee-4bb1-b45b-cd84608dd6fa None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:38.412309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e55ff5-8760-4d89-9396-7bc3065a9cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:38.413202 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e55ff5-8760-4d89-9396-7bc3065a9cbc None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:38.413664 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e55ff5-8760-4d89-9396-7bc3065a9cbc None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:38.413809 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e55ff5-8760-4d89-9396-7bc3065a9cbc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:38.440510 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e55ff5-8760-4d89-9396-7bc3065a9cbc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:38.442813 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e55ff5-8760-4d89-9396-7bc3065a9cbc None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:38.445460 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5476ffe-4ec9-48f4-bc20-75017037f584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:38.446284 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5476ffe-4ec9-48f4-bc20-75017037f584 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:38.448253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5476ffe-4ec9-48f4-bc20-75017037f584 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:38.448698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5476ffe-4ec9-48f4-bc20-75017037f584 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:38.449769 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5476ffe-4ec9-48f4-bc20-75017037f584 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:38.451161 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5476ffe-4ec9-48f4-bc20-75017037f584 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:38.469355 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-03e55ff5-8760-4d89-9396-7bc3065a9cbc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:38.469355 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:38.469355 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:38.469355 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:38.469355 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:38.469355 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:38.469355 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:38.469355 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:38.469355 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:38.469355 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:38.469355 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:38.469355 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:38.469355 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:38.469355 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:38.469355 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:38.470304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:38.470304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:38.470304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:38.470304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:38.470304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:38.470304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:38.470304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:38.470304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:38.470304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:38.470304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:38.470304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:38.470304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:38.470304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:38.470304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:38.470304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:38.470304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:38.470304 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:38.471234 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:38.471234 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:38.471234 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:38.471234 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:38.473657 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 414/827] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:38.480558 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e55ff5-8760-4d89-9396-7bc3065a9cbc None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:38.489284 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 414/828] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 23 16:54:38 2022] POST /identity/v3/endpoints => generated 346 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:54:38.611444 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:38.620265 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e55ff5-8760-4d89-9396-7bc3065a9cbc None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:38.686359 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63b01f-39e8-486a-8a09-98549aa1a0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:38.686989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63b01f-39e8-486a-8a09-98549aa1a0ba None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:38.687278 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63b01f-39e8-486a-8a09-98549aa1a0ba None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:38.687552 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63b01f-39e8-486a-8a09-98549aa1a0ba None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:38.688315 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63b01f-39e8-486a-8a09-98549aa1a0ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:38.688636 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63b01f-39e8-486a-8a09-98549aa1a0ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:38.690069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63b01f-39e8-486a-8a09-98549aa1a0ba None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:38.697273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1137526-e7cd-405d-8a6c-4e4513e71f15 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:38.697549 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1137526-e7cd-405d-8a6c-4e4513e71f15 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:38.697830 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1137526-e7cd-405d-8a6c-4e4513e71f15 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:38.699522 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 415/829] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:54:38.705365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283480d-2b97-4f86-a529-61854ba3b4ec None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:38.705652 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283480d-2b97-4f86-a529-61854ba3b4ec None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:38.705985 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283480d-2b97-4f86-a529-61854ba3b4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:38.709496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 415/830] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:38.715006 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63b01f-39e8-486a-8a09-98549aa1a0ba None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:38.726026 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-c283480d-2b97-4f86-a529-61854ba3b4ec None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:38.727397 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c283480d-2b97-4f86-a529-61854ba3b4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:38.815888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:38.822518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 416/831] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:38 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 16:54:38.844244 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63b01f-39e8-486a-8a09-98549aa1a0ba None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:38.845115 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283480d-2b97-4f86-a529-61854ba3b4ec None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:38.879466 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407bb302-1d69-459a-9c67-b0bf553e33f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:38.880287 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bb302-1d69-459a-9c67-b0bf553e33f1 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:38.880574 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bb302-1d69-459a-9c67-b0bf553e33f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:38.880850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bb302-1d69-459a-9c67-b0bf553e33f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:38.888628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407bb302-1d69-459a-9c67-b0bf553e33f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:38.890192 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407bb302-1d69-459a-9c67-b0bf553e33f1 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:38.905290 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-407bb302-1d69-459a-9c67-b0bf553e33f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:38.905290 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:38.905290 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:38.905290 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:38.905290 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:38.905290 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:38.905290 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:38.905290 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:38.905290 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:38.905290 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:38.905290 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:38.905290 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:38.905290 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:38.905290 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:38.905290 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:38.906303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:38.906303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:38.906303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:38.906303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:38.906303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:38.906303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:38.906303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:38.906303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:38.906303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:38.906303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:38.906303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:38.906303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:38.906303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:38.906303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:38.906303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:38.906303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:38.906303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:38.907272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:38.907272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:38.907272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:38.907272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:38.907272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:38.907272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:38.907272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:38.907272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:38.907272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 416/832] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:54:38.912346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407bb302-1d69-459a-9c67-b0bf553e33f1 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:38.927259 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:38.947644 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283480d-2b97-4f86-a529-61854ba3b4ec None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:39.000340 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763b980-9658-4775-b70a-573bbfd36505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:39.001517 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763b980-9658-4775-b70a-573bbfd36505 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:39.001935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763b980-9658-4775-b70a-573bbfd36505 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:39.002279 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4763b980-9658-4775-b70a-573bbfd36505 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:39.018718 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4763b980-9658-4775-b70a-573bbfd36505 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:39.023640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407bb302-1d69-459a-9c67-b0bf553e33f1 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:39.029138 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-4763b980-9658-4775-b70a-573bbfd36505 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:54:39.029138 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:39.029138 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:39.029138 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:39.029138 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:39.029138 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:39.029138 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:39.029138 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:39.029138 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:39.029138 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:39.029138 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:39.029138 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:39.029138 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:39.029138 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:39.029138 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:39.029138 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:39.030279 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:39.030279 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:39.030279 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:39.030279 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:39.030279 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:39.030279 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:39.030279 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:39.030279 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:39.030279 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:39.030279 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:39.030279 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:39.030279 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:39.030279 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:39.030279 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:39.030279 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:39.030279 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:39.031365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:39.031365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:39.031365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:39.031365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:39.031365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:39.031365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:39.031365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:39.031365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:39.031365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:39.031365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:39.031365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:54:39.031365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:39.033321 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 417/833] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:54:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:54:39.038971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763b980-9658-4775-b70a-573bbfd36505 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:39.061763 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bccaf53-c5b4-44d2-8992-2ffc0ee65779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:39.062434 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccaf53-c5b4-44d2-8992-2ffc0ee65779 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:39.062720 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccaf53-c5b4-44d2-8992-2ffc0ee65779 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:39.063027 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bccaf53-c5b4-44d2-8992-2ffc0ee65779 None admin] PATH_INFO: `/v3/projects` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:39.063820 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bccaf53-c5b4-44d2-8992-2ffc0ee65779 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:39.064134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bccaf53-c5b4-44d2-8992-2ffc0ee65779 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:39.065688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bccaf53-c5b4-44d2-8992-2ffc0ee65779 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:39.078743 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 417/834] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:39.084715 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bccaf53-c5b4-44d2-8992-2ffc0ee65779 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:39.117293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:39.157754 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:39.173067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4763b980-9658-4775-b70a-573bbfd36505 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:39.193802 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bccaf53-c5b4-44d2-8992-2ffc0ee65779 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:39.216797 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc08a3e-c12e-4059-9f64-eab0a2f9a11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:39.217869 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc08a3e-c12e-4059-9f64-eab0a2f9a11c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:39.218658 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc08a3e-c12e-4059-9f64-eab0a2f9a11c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:39.219363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc08a3e-c12e-4059-9f64-eab0a2f9a11c None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:39.220431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc08a3e-c12e-4059-9f64-eab0a2f9a11c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:39.221042 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc08a3e-c12e-4059-9f64-eab0a2f9a11c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:39.223270 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc08a3e-c12e-4059-9f64-eab0a2f9a11c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:39.229934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7f50c-de19-4e5f-885d-b9e9ecdfbcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:39.230721 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7f50c-de19-4e5f-885d-b9e9ecdfbcda None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:39.231144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7f50c-de19-4e5f-885d-b9e9ecdfbcda None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:39.231523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e7f50c-de19-4e5f-885d-b9e9ecdfbcda None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:39.232503 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7f50c-de19-4e5f-885d-b9e9ecdfbcda None admin] RBAC: Adding query filter params (role.id=17bbe7cc1c3641509de5e9dca8534102, scope.project.id=0595667fb2e04a3ca4716f669d6698b0, user.id=225c7b385f12445fab86b3b5606b1da1) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:39.232979 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7f50c-de19-4e5f-885d-b9e9ecdfbcda None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:39.234620 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e7f50c-de19-4e5f-885d-b9e9ecdfbcda None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:39.239988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 418/835] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:54:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:39.242036 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc08a3e-c12e-4059-9f64-eab0a2f9a11c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:39.248638 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 418/836] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:54:39 2022] GET /identity/v3/role_assignments?user.id=225c7b385f12445fab86b3b5606b1da1&scope.project.id=0595667fb2e04a3ca4716f669d6698b0&role.id=17bbe7cc1c3641509de5e9dca8534102 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:39.370901 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc08a3e-c12e-4059-9f64-eab0a2f9a11c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:39.397196 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:39.410713 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07fc090-8167-47b5-bf93-31d71c7ffb7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:39.411607 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fc090-8167-47b5-bf93-31d71c7ffb7b None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:39.412109 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fc090-8167-47b5-bf93-31d71c7ffb7b None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:39.412383 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07fc090-8167-47b5-bf93-31d71c7ffb7b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:39.424936 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07fc090-8167-47b5-bf93-31d71c7ffb7b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:39.426428 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07fc090-8167-47b5-bf93-31d71c7ffb7b None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:39.445929 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-a07fc090-8167-47b5-bf93-31d71c7ffb7b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:39.445929 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:39.445929 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:39.445929 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:39.445929 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:39.445929 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:39.445929 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:39.445929 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:39.445929 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:39.445929 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:39.445929 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:39.445929 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:39.445929 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:39.445929 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:39.445929 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:39.446991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:39.446991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:39.446991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:39.446991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:39.446991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:39.446991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:39.446991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:39.446991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:39.446991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:39.446991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:39.446991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:39.446991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:39.446991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:39.446991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:39.446991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:39.446991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:39.446991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:39.447920 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:39.447920 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:39.447920 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:39.447920 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:39.447920 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 419/837] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:39.454576 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7f50c-de19-4e5f-885d-b9e9ecdfbcda None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:39.570186 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e7f50c-de19-4e5f-885d-b9e9ecdfbcda None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:39.600330 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:39.605679 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13863f-001c-4a39-b5da-359517689042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:39.607344 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13863f-001c-4a39-b5da-359517689042 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:39.607344 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13863f-001c-4a39-b5da-359517689042 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:39.607344 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca13863f-001c-4a39-b5da-359517689042 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:39.607879 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13863f-001c-4a39-b5da-359517689042 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:39.608195 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13863f-001c-4a39-b5da-359517689042 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:39.609697 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca13863f-001c-4a39-b5da-359517689042 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:39.622989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 419/838] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:39.629543 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07fc090-8167-47b5-bf93-31d71c7ffb7b None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:39.733139 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07fc090-8167-47b5-bf93-31d71c7ffb7b None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:39.772729 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc37dc-3871-4b5a-bccc-50e0d039b1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:39.773994 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc37dc-3871-4b5a-bccc-50e0d039b1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:39.774468 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc37dc-3871-4b5a-bccc-50e0d039b1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:39.774982 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1fc37dc-3871-4b5a-bccc-50e0d039b1e5 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:39.784029 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc37dc-3871-4b5a-bccc-50e0d039b1e5 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:39.785856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1fc37dc-3871-4b5a-bccc-50e0d039b1e5 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:39.801343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-b1fc37dc-3871-4b5a-bccc-50e0d039b1e5 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 16:54:39.801343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:39.801343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:39.801343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:39.801343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:39.801343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:39.801343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:39.801343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:39.801343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:39.801343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:39.801343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:39.801343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:39.801343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:39.801343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:39.801343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:39.801343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:39.802348 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:39.802348 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:39.802348 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:39.802348 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:39.802348 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:39.802348 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:39.802348 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:39.802348 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:39.802348 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:39.802348 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:39.802348 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:39.802348 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:39.802348 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:39.802348 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:39.802348 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:39.802348 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:39.802348 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:39.802348 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:39.803345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:39.803345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:39.803345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:39.803345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:39.803345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:39.803345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:39.803345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:39.803345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:39.803345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 16:54:39.803345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:39.809866 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 420/839] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:54:39 2022] GET /identity/v3/users/swift => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:39.812690 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13863f-001c-4a39-b5da-359517689042 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:39.920511 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca13863f-001c-4a39-b5da-359517689042 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:39.957968 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba920d8a-7d69-4bce-87da-0a5bc0c339bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:39.958663 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba920d8a-7d69-4bce-87da-0a5bc0c339bd None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:39.959023 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba920d8a-7d69-4bce-87da-0a5bc0c339bd None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:39.959303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba920d8a-7d69-4bce-87da-0a5bc0c339bd None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:39.960157 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba920d8a-7d69-4bce-87da-0a5bc0c339bd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:39.960470 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba920d8a-7d69-4bce-87da-0a5bc0c339bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:39.961912 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba920d8a-7d69-4bce-87da-0a5bc0c339bd None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:39.981098 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 420/840] 10.222.0.13 () {56 vars in 1135 bytes} [Mon May 23 16:54:39 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:39.987029 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc37dc-3871-4b5a-bccc-50e0d039b1e5 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:40.023308 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:40.099729 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1fc37dc-3871-4b5a-bccc-50e0d039b1e5 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:40.135345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379ff2e-008b-4a82-a3a2-d08f15e33b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:40.136012 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379ff2e-008b-4a82-a3a2-d08f15e33b4c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:40.136293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379ff2e-008b-4a82-a3a2-d08f15e33b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:40.136563 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4379ff2e-008b-4a82-a3a2-d08f15e33b4c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:40.206060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379ff2e-008b-4a82-a3a2-d08f15e33b4c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:40.207576 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4379ff2e-008b-4a82-a3a2-d08f15e33b4c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:40.241056 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:40.270646 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3649956-464d-4cd0-86cf-eb84736d972e None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:40.271025 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3649956-464d-4cd0-86cf-eb84736d972e None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:40.271311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3649956-464d-4cd0-86cf-eb84736d972e None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:40.272837 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 421/841] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:40.278189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47c493-fcb3-4e7b-a687-60c0cff200e8 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:40.278481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47c493-fcb3-4e7b-a687-60c0cff200e8 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:40.278764 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c47c493-fcb3-4e7b-a687-60c0cff200e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:40.301690 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-5c47c493-fcb3-4e7b-a687-60c0cff200e8 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:40.304027 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-5c47c493-fcb3-4e7b-a687-60c0cff200e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:40.309669 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 421/842] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:54:39 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 16:54:40.315005 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379ff2e-008b-4a82-a3a2-d08f15e33b4c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:40.357346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:40.381527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 422/843] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:40 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:54:40.405699 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47c493-fcb3-4e7b-a687-60c0cff200e8 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:40.430009 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4379ff2e-008b-4a82-a3a2-d08f15e33b4c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:40.468722 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc43214-4d9e-4b1b-862b-5e8836661db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:40.469836 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc43214-4d9e-4b1b-862b-5e8836661db1 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:40.470293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc43214-4d9e-4b1b-862b-5e8836661db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:40.470855 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc43214-4d9e-4b1b-862b-5e8836661db1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:40.473561 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:40.486531 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc43214-4d9e-4b1b-862b-5e8836661db1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:40.488325 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc43214-4d9e-4b1b-862b-5e8836661db1 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:40.509051 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-0cc43214-4d9e-4b1b-862b-5e8836661db1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:40.509051 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:40.509051 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:40.509051 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:40.509051 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:40.509051 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:40.509051 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:40.509051 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:40.509051 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:40.509051 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:40.509051 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:40.509051 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:40.509051 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:40.509051 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:40.509051 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:40.509973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:40.509973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:40.509973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:40.509973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:40.509973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:40.509973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:40.509973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:40.509973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:40.509973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:40.509973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:40.509973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:40.509973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:40.509973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:40.509973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:40.509973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:40.509973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:40.509973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:40.510855 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:40.510855 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:40.510855 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:40.510855 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:40.513300 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 422/844] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:40.518974 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc43214-4d9e-4b1b-862b-5e8836661db1 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:40.528079 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c47c493-fcb3-4e7b-a687-60c0cff200e8 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:40.577694 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95abab1-6373-4978-b3ca-dafe31d623d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:40.579157 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95abab1-6373-4978-b3ca-dafe31d623d1 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:40.579517 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95abab1-6373-4978-b3ca-dafe31d623d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:40.579870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95abab1-6373-4978-b3ca-dafe31d623d1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:40.595265 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95abab1-6373-4978-b3ca-dafe31d623d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:40.606987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-f95abab1-6373-4978-b3ca-dafe31d623d1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:40.606987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:40.606987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:40.606987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:40.606987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:40.606987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:40.606987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:40.606987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:40.606987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:40.606987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:40.606987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:40.606987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:40.606987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:40.606987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:40.606987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:40.606987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:40.607985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:40.607985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:40.607985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:40.607985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:40.607985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:40.607985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:40.607985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:40.607985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:40.607985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:40.607985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:40.607985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:40.607985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:40.607985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:40.607985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:40.607985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:40.607985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:40.608817 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:40.608817 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:40.608817 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:40.608817 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:40.608817 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:40.608817 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:40.608817 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:40.608817 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:40.608817 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:40.608817 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:40.608817 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 16:54:40.608817 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:40.610991 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 423/845] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 23 16:54:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:54:40.615890 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95abab1-6373-4978-b3ca-dafe31d623d1 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:40.638814 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc43214-4d9e-4b1b-862b-5e8836661db1 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:40.672158 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:40.675789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf1e36-367a-4b92-9ec5-349ef6ac64c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:40.676807 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf1e36-367a-4b92-9ec5-349ef6ac64c6 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:40.677284 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf1e36-367a-4b92-9ec5-349ef6ac64c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:40.677723 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bf1e36-367a-4b92-9ec5-349ef6ac64c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:40.678694 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf1e36-367a-4b92-9ec5-349ef6ac64c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:40.679211 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf1e36-367a-4b92-9ec5-349ef6ac64c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:40.680798 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bf1e36-367a-4b92-9ec5-349ef6ac64c6 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:40.694419 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 423/846] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:40.699595 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf1e36-367a-4b92-9ec5-349ef6ac64c6 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:40.752499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95abab1-6373-4978-b3ca-dafe31d623d1 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:40.755005 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:40.800607 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cdc0ea-3d7f-409b-a3fe-96e9ff5e584f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:40.801369 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cdc0ea-3d7f-409b-a3fe-96e9ff5e584f None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:40.801773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cdc0ea-3d7f-409b-a3fe-96e9ff5e584f None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:40.802416 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02cdc0ea-3d7f-409b-a3fe-96e9ff5e584f None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:40.803317 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cdc0ea-3d7f-409b-a3fe-96e9ff5e584f None admin] RBAC: Adding query filter params (name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:40.803732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cdc0ea-3d7f-409b-a3fe-96e9ff5e584f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:40.806228 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02cdc0ea-3d7f-409b-a3fe-96e9ff5e584f None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:40.813872 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bf1e36-367a-4b92-9ec5-349ef6ac64c6 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:40.823660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 424/847] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 23 16:54:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:40.824836 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cdc0ea-3d7f-409b-a3fe-96e9ff5e584f None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:40.861155 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596df17-df2d-40fa-90c0-36e8a7240a63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:40.861964 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596df17-df2d-40fa-90c0-36e8a7240a63 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:40.862356 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596df17-df2d-40fa-90c0-36e8a7240a63 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:40.862709 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596df17-df2d-40fa-90c0-36e8a7240a63 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:40.870415 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0596df17-df2d-40fa-90c0-36e8a7240a63 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:40.871958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0596df17-df2d-40fa-90c0-36e8a7240a63 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:40.886997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-0596df17-df2d-40fa-90c0-36e8a7240a63 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:40.886997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:40.886997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:40.886997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:40.886997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:40.886997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:40.886997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:40.886997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:40.886997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:40.886997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:40.886997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:40.886997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:40.886997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:40.886997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:40.886997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:40.887947 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:40.887947 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:40.887947 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:40.887947 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:40.887947 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:40.887947 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:40.887947 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:40.887947 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:40.887947 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:40.887947 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:40.887947 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:40.887947 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:40.887947 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:40.887947 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:40.887947 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:40.887947 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:40.887947 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:40.888835 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:40.888835 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:40.888835 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:40.888835 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:40.888835 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:40.888835 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:40.888835 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:40.888835 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:40.893074 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 424/848] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:40.894751 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596df17-df2d-40fa-90c0-36e8a7240a63 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:40.949309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02cdc0ea-3d7f-409b-a3fe-96e9ff5e584f None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:40.950978 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:40.985861 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75848aff-9558-40bc-a3b3-2d3c3344e7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:40.986932 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75848aff-9558-40bc-a3b3-2d3c3344e7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:40.987543 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75848aff-9558-40bc-a3b3-2d3c3344e7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:40.987873 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75848aff-9558-40bc-a3b3-2d3c3344e7c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:40.998871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75848aff-9558-40bc-a3b3-2d3c3344e7c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:40.999468 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596df17-df2d-40fa-90c0-36e8a7240a63 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:41.000338 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75848aff-9558-40bc-a3b3-2d3c3344e7c3 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:41.007361 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:41.024706 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-75848aff-9558-40bc-a3b3-2d3c3344e7c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:41.024706 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:41.024706 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:41.024706 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:41.024706 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:41.024706 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:41.024706 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:41.024706 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:41.024706 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:41.024706 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:41.024706 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:41.024706 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:41.024706 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:41.024706 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:41.024706 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:41.025692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:41.025692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:41.025692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:41.025692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:41.025692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:41.025692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:41.025692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:41.025692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:41.025692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:41.025692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:41.025692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:41.025692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:41.025692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:41.025692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:41.025692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:41.025692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:41.025692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:41.026606 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:41.026606 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:41.026606 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:41.026606 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:41.031060 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 425/849] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:41.032685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84a3c5-1c91-4ce6-b38e-094188bfc7cb None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:41.032980 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84a3c5-1c91-4ce6-b38e-094188bfc7cb None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:41.033246 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b84a3c5-1c91-4ce6-b38e-094188bfc7cb None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:41.035603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 426/850] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:41.036808 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84a3c5-1c91-4ce6-b38e-094188bfc7cb None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:41.043218 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-827ec75e-965f-46dd-8125-038517177713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:41.043922 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ec75e-965f-46dd-8125-038517177713 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:41.044209 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ec75e-965f-46dd-8125-038517177713 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:41.044487 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827ec75e-965f-46dd-8125-038517177713 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:41.045305 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ec75e-965f-46dd-8125-038517177713 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:41.045612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ec75e-965f-46dd-8125-038517177713 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:41.047611 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-827ec75e-965f-46dd-8125-038517177713 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:41.069869 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 425/851] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:41.069869 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb91173-4530-4e7c-807b-e19ee7ca1dd9 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:41.069869 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb91173-4530-4e7c-807b-e19ee7ca1dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:41.069869 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb91173-4530-4e7c-807b-e19ee7ca1dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:41.087279 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-1bb91173-4530-4e7c-807b-e19ee7ca1dd9 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:41.088473 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-1bb91173-4530-4e7c-807b-e19ee7ca1dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:41.151349 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b84a3c5-1c91-4ce6-b38e-094188bfc7cb None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:41.174977 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 426/852] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:54:41.174977 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb91173-4530-4e7c-807b-e19ee7ca1dd9 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:41.185765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5b764-5397-464d-8c84-4ae1955f9973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:41.186446 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5b764-5397-464d-8c84-4ae1955f9973 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:41.186729 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5b764-5397-464d-8c84-4ae1955f9973 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:41.187038 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b5b764-5397-464d-8c84-4ae1955f9973 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:41.187783 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5b764-5397-464d-8c84-4ae1955f9973 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:41.188081 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5b764-5397-464d-8c84-4ae1955f9973 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:41.189469 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b5b764-5397-464d-8c84-4ae1955f9973 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:41.202411 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 427/853] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:41.203467 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5b764-5397-464d-8c84-4ae1955f9973 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:41.221207 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:41.292249 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:41.298255 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb91173-4530-4e7c-807b-e19ee7ca1dd9 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:41.327726 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b5b764-5397-464d-8c84-4ae1955f9973 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:41.345186 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523daafb-d084-40aa-8fc6-1595838a4b75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:41.345856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523daafb-d084-40aa-8fc6-1595838a4b75 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:41.346137 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523daafb-d084-40aa-8fc6-1595838a4b75 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:41.346427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523daafb-d084-40aa-8fc6-1595838a4b75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:41.347308 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523daafb-d084-40aa-8fc6-1595838a4b75 None admin] RBAC: Adding query filter params (role.id=17bbe7cc1c3641509de5e9dca8534102, scope.project.id=0595667fb2e04a3ca4716f669d6698b0, user.id=6546217667054abb8eb07076995c1b04) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:41.347615 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523daafb-d084-40aa-8fc6-1595838a4b75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:41.349763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523daafb-d084-40aa-8fc6-1595838a4b75 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:41.361590 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c4091-26d5-409f-b6ea-2e6c79c0caa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:41.362586 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c4091-26d5-409f-b6ea-2e6c79c0caa1 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:41.362905 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c4091-26d5-409f-b6ea-2e6c79c0caa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:41.363236 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00c4091-26d5-409f-b6ea-2e6c79c0caa1 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:41.363881 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00c4091-26d5-409f-b6ea-2e6c79c0caa1 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:41.365257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00c4091-26d5-409f-b6ea-2e6c79c0caa1 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:41.367653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 427/854] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:54:41 2022] GET /identity/v3/role_assignments?user.id=6546217667054abb8eb07076995c1b04&scope.project.id=0595667fb2e04a3ca4716f669d6698b0&role.id=17bbe7cc1c3641509de5e9dca8534102 => generated 608 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:41.369548 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523daafb-d084-40aa-8fc6-1595838a4b75 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:41.378407 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-f00c4091-26d5-409f-b6ea-2e6c79c0caa1 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 16:54:41.378407 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:41.378407 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:54:41.378407 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:54:41.378407 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:54:41.378407 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:54:41.378407 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:54:41.378407 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:41.378407 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 16:54:41.378407 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:41.378407 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:54:41.378407 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:41.378407 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:41.378407 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:41.378407 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:41.378407 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:41.378407 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:41.380230 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:41.380230 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:41.380230 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:41.380230 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:41.380230 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:41.380230 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:41.380230 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:54:41.380230 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:54:41.380230 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:54:41.380230 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:54:41.380230 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:41.380230 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:41.380230 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:41.380230 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:41.380230 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:41.380230 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:41.381972 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:41.381972 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:41.381972 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:41.381972 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:41.381972 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:41.381972 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:41.381972 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:41.381972 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:41.381972 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:41.381972 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:41.381972 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:54:41.381972 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:41.381972 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 16:54:41.381972 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:41.385542 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 428/855] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 23 16:54:41 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:41.386575 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c4091-26d5-409f-b6ea-2e6c79c0caa1 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:41.471950 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:41.503068 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00c4091-26d5-409f-b6ea-2e6c79c0caa1 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:41.525321 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523daafb-d084-40aa-8fc6-1595838a4b75 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:41.542594 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6053c-0fde-4de0-b7a3-ac4574337d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:41.543704 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6053c-0fde-4de0-b7a3-ac4574337d99 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:41.544190 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6053c-0fde-4de0-b7a3-ac4574337d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:41.544662 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f6053c-0fde-4de0-b7a3-ac4574337d99 None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:41.545624 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6053c-0fde-4de0-b7a3-ac4574337d99 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:41.546174 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6053c-0fde-4de0-b7a3-ac4574337d99 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:41.547828 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f6053c-0fde-4de0-b7a3-ac4574337d99 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:41.558766 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 429/856] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 23 16:54:41 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:41.560120 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6053c-0fde-4de0-b7a3-ac4574337d99 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:41.573189 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573d4e8-6203-4fbd-bc47-782586225062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:41.574202 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573d4e8-6203-4fbd-bc47-782586225062 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:41.574534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573d4e8-6203-4fbd-bc47-782586225062 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:41.574865 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0573d4e8-6203-4fbd-bc47-782586225062 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:41.586075 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573d4e8-6203-4fbd-bc47-782586225062 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:41.590459 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0573d4e8-6203-4fbd-bc47-782586225062 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:41.609847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-0573d4e8-6203-4fbd-bc47-782586225062 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 16:54:41.609847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:41.609847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:41.609847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:41.609847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:41.609847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:41.609847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:41.609847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:41.609847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:41.609847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:41.609847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:41.609847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:41.609847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:41.609847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:41.609847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:41.609847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:41.610841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:41.610841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:41.610841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:41.610841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:41.610841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:41.610841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:41.610841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:41.610841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:41.610841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:41.610841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:41.610841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:41.610841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:41.610841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:41.610841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:41.610841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:41.610841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:41.610841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:41.610841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:41.611813 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:41.611813 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:41.611813 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:41.611813 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:41.611813 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:41.611813 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:41.611813 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:41.611813 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:41.611813 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 16:54:41.611813 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:41.611813 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 428/857] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 23 16:54:41 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:41.620590 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573d4e8-6203-4fbd-bc47-782586225062 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:41.653140 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:41.699170 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f6053c-0fde-4de0-b7a3-ac4574337d99 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:41.735628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2670c570-55d5-44b4-8761-ca71760045fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:41.736537 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670c570-55d5-44b4-8761-ca71760045fb None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:41.736868 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670c570-55d5-44b4-8761-ca71760045fb None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:41.737276 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2670c570-55d5-44b4-8761-ca71760045fb None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:41.738351 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2670c570-55d5-44b4-8761-ca71760045fb None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:41.738663 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2670c570-55d5-44b4-8761-ca71760045fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:41.740107 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2670c570-55d5-44b4-8761-ca71760045fb None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:41.750948 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 430/858] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 23 16:54:41 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:41.750948 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4457a4c-3729-404f-a120-d292fce5b666 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:41.750948 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4457a4c-3729-404f-a120-d292fce5b666 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:41.750948 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4457a4c-3729-404f-a120-d292fce5b666 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:41.750948 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 431/859] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:41.754233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0573d4e8-6203-4fbd-bc47-782586225062 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:41.754987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8010a8-580d-4b98-a412-942fbc6eac5b None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:41.755321 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8010a8-580d-4b98-a412-942fbc6eac5b None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:41.755698 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8010a8-580d-4b98-a412-942fbc6eac5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:41.769964 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:41.775799 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-2d8010a8-580d-4b98-a412-942fbc6eac5b None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:41.778834 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-2d8010a8-580d-4b98-a412-942fbc6eac5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:41.794095 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4acd05-19d8-4fcf-8e9d-a568904d5c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:41.795006 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4acd05-19d8-4fcf-8e9d-a568904d5c4e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:41.795390 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4acd05-19d8-4fcf-8e9d-a568904d5c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:41.795759 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4acd05-19d8-4fcf-8e9d-a568904d5c4e None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:41.796825 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4acd05-19d8-4fcf-8e9d-a568904d5c4e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:41.797326 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4acd05-19d8-4fcf-8e9d-a568904d5c4e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:41.799288 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4acd05-19d8-4fcf-8e9d-a568904d5c4e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:41.821621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 429/860] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 23 16:54:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:41.827571 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4acd05-19d8-4fcf-8e9d-a568904d5c4e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:41.883390 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 432/861] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:41 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 16:54:41.910028 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8010a8-580d-4b98-a412-942fbc6eac5b None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:41.932889 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:41.962118 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4acd05-19d8-4fcf-8e9d-a568904d5c4e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:42.008483 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1dc93-e105-453e-a8cd-3f9b2ff29b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:42.009638 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1dc93-e105-453e-a8cd-3f9b2ff29b15 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:42.009959 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1dc93-e105-453e-a8cd-3f9b2ff29b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:42.010282 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1dc93-e105-453e-a8cd-3f9b2ff29b15 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:42.039058 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8010a8-580d-4b98-a412-942fbc6eac5b None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:42.078809 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfab2df-b615-448d-bf52-6270613dc3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:42.079888 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfab2df-b615-448d-bf52-6270613dc3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:42.080203 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfab2df-b615-448d-bf52-6270613dc3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:42.080498 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dfab2df-b615-448d-bf52-6270613dc3d4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:42.081345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfab2df-b615-448d-bf52-6270613dc3d4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:42.082934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dfab2df-b615-448d-bf52-6270613dc3d4 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:42.086230 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:42.098485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-1dfab2df-b615-448d-bf52-6270613dc3d4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 16:54:42.098485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:42.098485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:54:42.098485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:54:42.098485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:54:42.098485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:54:42.098485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:54:42.098485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:42.098485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 16:54:42.098485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:42.098485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:54:42.098485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:42.098485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:42.098485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:42.098485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:42.098485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:42.099639 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:42.099639 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:42.099639 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:42.099639 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:42.099639 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:42.099639 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:42.099639 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:42.099639 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:54:42.099639 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:54:42.099639 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:54:42.099639 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:54:42.099639 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:42.099639 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:42.099639 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:42.099639 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:42.099639 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:42.100474 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:42.100474 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:42.100474 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:42.100474 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:42.100474 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:42.100474 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:42.100474 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:42.100474 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:42.100474 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:42.100474 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:42.100474 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:42.100474 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:54:42.100474 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:42.100474 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 16:54:42.100474 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:42.101316 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b1dc93-e105-453e-a8cd-3f9b2ff29b15 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:42.102011 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b1dc93-e105-453e-a8cd-3f9b2ff29b15 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:42.105618 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 433/862] 10.222.0.13 () {56 vars in 1132 bytes} [Mon May 23 16:54:41 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:54:42.117288 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfab2df-b615-448d-bf52-6270613dc3d4 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:42.198896 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:42.236478 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 430/863] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:54:41 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 16:54:42.237871 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1dc93-e105-453e-a8cd-3f9b2ff29b15 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:42.242364 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dfab2df-b615-448d-bf52-6270613dc3d4 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:42.284118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c65e4b-8dd1-45bb-8076-46afa0aa35bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:42.285072 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c65e4b-8dd1-45bb-8076-46afa0aa35bc None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:42.285977 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c65e4b-8dd1-45bb-8076-46afa0aa35bc None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:42.286395 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c65e4b-8dd1-45bb-8076-46afa0aa35bc None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:42.287385 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c65e4b-8dd1-45bb-8076-46afa0aa35bc None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:42.288294 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c65e4b-8dd1-45bb-8076-46afa0aa35bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:42.290657 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c65e4b-8dd1-45bb-8076-46afa0aa35bc None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:42.297868 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 434/864] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 23 16:54:42 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:42.306226 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c65e4b-8dd1-45bb-8076-46afa0aa35bc None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:42.364971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1dc93-e105-453e-a8cd-3f9b2ff29b15 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:42.379640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:42.415418 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549b445-bf36-44cb-b3bc-35a53d60a690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:42.416320 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549b445-bf36-44cb-b3bc-35a53d60a690 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:42.416757 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549b445-bf36-44cb-b3bc-35a53d60a690 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:42.417032 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a549b445-bf36-44cb-b3bc-35a53d60a690 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:42.423639 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c65e4b-8dd1-45bb-8076-46afa0aa35bc None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:42.429497 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549b445-bf36-44cb-b3bc-35a53d60a690 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:42.431003 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a549b445-bf36-44cb-b3bc-35a53d60a690 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:42.452797 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-a549b445-bf36-44cb-b3bc-35a53d60a690 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:42.452797 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:42.452797 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:42.452797 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:42.452797 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:42.452797 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:42.452797 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:42.452797 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:42.452797 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:42.452797 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:42.452797 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:42.452797 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:42.452797 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:42.452797 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:42.452797 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:42.453693 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:42.453693 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:42.453693 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:42.453693 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:42.453693 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:42.453693 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:42.453693 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:42.453693 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:42.453693 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:42.453693 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:42.453693 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:42.453693 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:42.453693 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:42.453693 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:42.453693 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:42.453693 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:42.453693 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:42.454562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:42.454562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:42.454562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:42.454562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:42.459476 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 431/865] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:42.460319 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549b445-bf36-44cb-b3bc-35a53d60a690 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:42.467595 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78db706-bd68-4f08-b80a-6da932a5b45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:42.468531 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78db706-bd68-4f08-b80a-6da932a5b45d None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:42.468910 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78db706-bd68-4f08-b80a-6da932a5b45d None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:42.469281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78db706-bd68-4f08-b80a-6da932a5b45d None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:42.470511 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78db706-bd68-4f08-b80a-6da932a5b45d None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:42.470941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78db706-bd68-4f08-b80a-6da932a5b45d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:42.473379 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78db706-bd68-4f08-b80a-6da932a5b45d None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:42.491363 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 435/866] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 23 16:54:42 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:42.496363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:42.580602 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a549b445-bf36-44cb-b3bc-35a53d60a690 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:42.624792 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a6bae-5db5-4093-b7d4-de1ba271c58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:42.626037 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a6bae-5db5-4093-b7d4-de1ba271c58d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:42.626576 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a6bae-5db5-4093-b7d4-de1ba271c58d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:42.627086 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0a6bae-5db5-4093-b7d4-de1ba271c58d None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:42.628353 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a6bae-5db5-4093-b7d4-de1ba271c58d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:42.629027 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a6bae-5db5-4093-b7d4-de1ba271c58d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:42.630688 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d0a6bae-5db5-4093-b7d4-de1ba271c58d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:42.646239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 432/867] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:42.652095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78db706-bd68-4f08-b80a-6da932a5b45d None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:42.764363 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78db706-bd68-4f08-b80a-6da932a5b45d None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:42.792850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:42.801215 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651d022-55ab-43bc-b8d5-c01890c004e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:42.802322 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651d022-55ab-43bc-b8d5-c01890c004e6 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:42.802981 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651d022-55ab-43bc-b8d5-c01890c004e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:42.803645 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2651d022-55ab-43bc-b8d5-c01890c004e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:42.810808 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2651d022-55ab-43bc-b8d5-c01890c004e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:42.812385 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2651d022-55ab-43bc-b8d5-c01890c004e6 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:42.827114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-2651d022-55ab-43bc-b8d5-c01890c004e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:42.827114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:42.827114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:42.827114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:42.827114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:42.827114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:42.827114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:42.827114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:42.827114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:42.827114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:42.827114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:42.827114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:42.827114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:42.827114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:42.827114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:42.828167 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:42.828167 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:42.828167 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:42.828167 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:42.828167 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:42.828167 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:42.828167 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:42.828167 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:42.828167 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:42.828167 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:42.828167 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:42.828167 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:42.828167 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:42.828167 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:42.828167 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:42.828167 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:42.828167 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:42.829259 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:42.829259 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:42.829259 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:42.829259 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:42.829259 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:42.829259 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:42.829259 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:42.829259 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:42.829259 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 436/868] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:42.834645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a6bae-5db5-4093-b7d4-de1ba271c58d None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:42.930562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:42.941990 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d0a6bae-5db5-4093-b7d4-de1ba271c58d None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:42.977095 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e79f0139-f69b-4ff9-b378-b1bb3a32c1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:42.977774 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f0139-f69b-4ff9-b378-b1bb3a32c1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:42.978056 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f0139-f69b-4ff9-b378-b1bb3a32c1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:42.978330 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e79f0139-f69b-4ff9-b378-b1bb3a32c1f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:42.979193 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f0139-f69b-4ff9-b378-b1bb3a32c1f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:42.979502 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f0139-f69b-4ff9-b378-b1bb3a32c1f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:42.980981 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e79f0139-f69b-4ff9-b378-b1bb3a32c1f0 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:42.995461 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 433/869] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:43.001535 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651d022-55ab-43bc-b8d5-c01890c004e6 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:43.111550 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2651d022-55ab-43bc-b8d5-c01890c004e6 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:43.153099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a89e9ad-f78f-4e04-a4b9-0fe3b261de3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:43.154252 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89e9ad-f78f-4e04-a4b9-0fe3b261de3f None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:43.154760 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89e9ad-f78f-4e04-a4b9-0fe3b261de3f None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:43.155262 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a89e9ad-f78f-4e04-a4b9-0fe3b261de3f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:43.156299 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89e9ad-f78f-4e04-a4b9-0fe3b261de3f None admin] RBAC: Adding query filter params (role.id=5a7214dc069f45969e19c44122b378a4, scope.project.id=0595667fb2e04a3ca4716f669d6698b0, user.id=4b2d638e6bd240ccbe45228352900bdb) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:43.156817 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89e9ad-f78f-4e04-a4b9-0fe3b261de3f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:43.158553 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a89e9ad-f78f-4e04-a4b9-0fe3b261de3f None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:43.170938 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 437/870] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:54:43 2022] GET /identity/v3/role_assignments?user.id=4b2d638e6bd240ccbe45228352900bdb&scope.project.id=0595667fb2e04a3ca4716f669d6698b0&role.id=5a7214dc069f45969e19c44122b378a4 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:43.759010 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d1a6c-10ab-41e5-8aa2-75c3234af04f None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:43.759500 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d1a6c-10ab-41e5-8aa2-75c3234af04f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:43.759500 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d1a6c-10ab-41e5-8aa2-75c3234af04f None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:43.761111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 434/871] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:43.766649 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1764be1-9971-4646-ab42-322c7be13e46 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:43.767103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1764be1-9971-4646-ab42-322c7be13e46 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:43.767244 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1764be1-9971-4646-ab42-322c7be13e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:43.785728 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-c1764be1-9971-4646-ab42-322c7be13e46 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:43.786901 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-c1764be1-9971-4646-ab42-322c7be13e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:43.836614 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:43.863019 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 438/872] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:43 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:54:43.880909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d1a6c-10ab-41e5-8aa2-75c3234af04f None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:43.989567 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d1a6c-10ab-41e5-8aa2-75c3234af04f None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:44.028232 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce7937-0efd-481d-92a7-85d89e2ec4c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:44.029429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce7937-0efd-481d-92a7-85d89e2ec4c0 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:44.029910 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce7937-0efd-481d-92a7-85d89e2ec4c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:44.030420 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce7937-0efd-481d-92a7-85d89e2ec4c0 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:44.031325 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ce7937-0efd-481d-92a7-85d89e2ec4c0 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:44.032958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ce7937-0efd-481d-92a7-85d89e2ec4c0 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:44.046763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-14ce7937-0efd-481d-92a7-85d89e2ec4c0 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 16:54:44.046763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:44.046763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:54:44.046763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:54:44.046763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:54:44.046763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:54:44.046763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:54:44.046763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:44.046763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 16:54:44.046763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:44.046763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:54:44.046763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:44.046763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:44.046763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:44.046763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:44.046763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:44.046763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:44.047932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:44.047932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:44.047932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:44.047932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:44.047932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:44.047932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:44.047932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:54:44.047932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:54:44.047932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:54:44.047932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:54:44.047932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:44.047932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:44.047932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:44.047932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:44.047932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:44.047932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:44.048795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:44.048795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:44.048795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:44.048795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:44.048795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:44.048795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:44.048795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:44.048795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:44.048795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:44.048795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:44.048795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:54:44.048795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:44.048795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 16:54:44.048795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:44.051790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 435/873] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 23 16:54:43 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:44.058960 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1764be1-9971-4646-ab42-322c7be13e46 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:44.162249 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1764be1-9971-4646-ab42-322c7be13e46 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:44.197734 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3046b149-864c-4b9a-9204-7a4c88570d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:44.198418 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046b149-864c-4b9a-9204-7a4c88570d06 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:44.198701 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046b149-864c-4b9a-9204-7a4c88570d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:44.199007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3046b149-864c-4b9a-9204-7a4c88570d06 None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:44.199720 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3046b149-864c-4b9a-9204-7a4c88570d06 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:44.200023 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3046b149-864c-4b9a-9204-7a4c88570d06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:44.200963 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a1d43-c116-4c96-aa32-b88ed96049f2 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:44.201246 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a1d43-c116-4c96-aa32-b88ed96049f2 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:44.201463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3046b149-864c-4b9a-9204-7a4c88570d06 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:44.201562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec6a1d43-c116-4c96-aa32-b88ed96049f2 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:44.203141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 436/874] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:54:44.207039 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 439/875] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 23 16:54:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:44.208101 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0c929-722d-4846-93a2-c637bf07b193 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:44.208389 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0c929-722d-4846-93a2-c637bf07b193 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:44.208663 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0c929-722d-4846-93a2-c637bf07b193 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:44.215845 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3046b149-864c-4b9a-9204-7a4c88570d06 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:44.226651 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-9dc0c929-722d-4846-93a2-c637bf07b193 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:44.227937 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9dc0c929-722d-4846-93a2-c637bf07b193 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:44.238198 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:44.270690 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:44.316446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 437/876] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:44 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 16:54:44.329610 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3046b149-864c-4b9a-9204-7a4c88570d06 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:44.338820 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0c929-722d-4846-93a2-c637bf07b193 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:44.371955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996a7ad-5631-4c22-b6cb-637175b67c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:44.373035 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996a7ad-5631-4c22-b6cb-637175b67c48 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:44.373593 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996a7ad-5631-4c22-b6cb-637175b67c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:44.374136 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996a7ad-5631-4c22-b6cb-637175b67c48 None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:44.375056 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996a7ad-5631-4c22-b6cb-637175b67c48 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:44.375608 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996a7ad-5631-4c22-b6cb-637175b67c48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:44.377280 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0996a7ad-5631-4c22-b6cb-637175b67c48 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:44.383083 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 440/877] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 23 16:54:44 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:44.458957 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0c929-722d-4846-93a2-c637bf07b193 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:44.508905 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0265050-55f6-4dde-bf25-bba6308e7602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:44.509989 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0265050-55f6-4dde-bf25-bba6308e7602 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:44.510471 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0265050-55f6-4dde-bf25-bba6308e7602 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:44.510963 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0265050-55f6-4dde-bf25-bba6308e7602 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:44.511906 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0265050-55f6-4dde-bf25-bba6308e7602 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:44.513543 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0265050-55f6-4dde-bf25-bba6308e7602 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:44.530304 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 438/878] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 23 16:54:44 2022] POST /identity/v3/services => generated 251 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:54:44.862055 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb0ba6-5370-4f13-a8a2-f42434a65cf2 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:44.862795 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb0ba6-5370-4f13-a8a2-f42434a65cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:44.862795 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bb0ba6-5370-4f13-a8a2-f42434a65cf2 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:44.864252 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 441/879] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:44.869526 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153562c-7a7a-44f5-b640-c6b69206452b None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:44.869981 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153562c-7a7a-44f5-b640-c6b69206452b None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:44.870062 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153562c-7a7a-44f5-b640-c6b69206452b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:44.876173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:44.890221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-5153562c-7a7a-44f5-b640-c6b69206452b None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:44.891434 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5153562c-7a7a-44f5-b640-c6b69206452b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:44.969307 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 439/880] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:44 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:54:44.991861 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb0ba6-5370-4f13-a8a2-f42434a65cf2 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:45.098219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bb0ba6-5370-4f13-a8a2-f42434a65cf2 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:45.109800 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:45.135991 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3df5497-c810-49fd-83c2-be13f6ffbf03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:45.136666 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df5497-c810-49fd-83c2-be13f6ffbf03 None admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:45.136949 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df5497-c810-49fd-83c2-be13f6ffbf03 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:45.137221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3df5497-c810-49fd-83c2-be13f6ffbf03 None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:45.137944 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3df5497-c810-49fd-83c2-be13f6ffbf03 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:45.139333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3df5497-c810-49fd-83c2-be13f6ffbf03 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:45.155874 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 442/881] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 23 16:54:44 2022] POST /identity/v3/services => generated 269 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:54:45.553577 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c13ba-e6fa-4fa2-97ba-da1d28216d6d None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:45.553872 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c13ba-e6fa-4fa2-97ba-da1d28216d6d None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:45.554140 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-720c13ba-e6fa-4fa2-97ba-da1d28216d6d None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:45.555771 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 440/882] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:45.561159 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e086db-b4e5-44ed-9421-186219332a5c None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:45.561446 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e086db-b4e5-44ed-9421-186219332a5c None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:45.561719 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e086db-b4e5-44ed-9421-186219332a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:45.580169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-82e086db-b4e5-44ed-9421-186219332a5c None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:45.581958 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-82e086db-b4e5-44ed-9421-186219332a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:45.661055 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 443/883] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:45 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:54:45.684731 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c13ba-e6fa-4fa2-97ba-da1d28216d6d None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:45.793597 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-720c13ba-e6fa-4fa2-97ba-da1d28216d6d None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:45.822645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:45.828569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4e2e6-c3ac-4556-a914-b025607f2231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:45.829519 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4e2e6-c3ac-4556-a914-b025607f2231 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:45.829801 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4e2e6-c3ac-4556-a914-b025607f2231 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:45.830136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a4e2e6-c3ac-4556-a914-b025607f2231 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:45.846820 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a4e2e6-c3ac-4556-a914-b025607f2231 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:45.855052 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-13a4e2e6-c3ac-4556-a914-b025607f2231 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 16:54:45.855052 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:45.855052 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:45.855052 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:45.855052 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:45.855052 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:45.855052 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:45.855052 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:45.855052 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:45.855052 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:45.855052 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:45.855052 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:45.855052 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:45.855052 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:45.855052 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:45.855052 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:45.856043 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:45.856043 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:45.856043 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:45.856043 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:45.856043 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:45.856043 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:45.856043 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:45.856043 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:45.856043 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:45.856043 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:45.856043 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:45.856043 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:45.856043 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:45.856043 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:45.856043 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:45.856043 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:45.856974 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:45.856974 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:45.856974 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:45.856974 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:45.856974 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:45.856974 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:45.856974 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:45.856974 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:45.856974 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:45.856974 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:45.856974 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 16:54:45.856974 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:45.856974 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 441/884] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 23 16:54:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:45.863206 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e086db-b4e5-44ed-9421-186219332a5c None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:45.960262 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:45.965345 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e086db-b4e5-44ed-9421-186219332a5c None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:46.002072 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281bedb-4754-4c64-9aa5-6315a23122fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:46.003793 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281bedb-4754-4c64-9aa5-6315a23122fe None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:46.004272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281bedb-4754-4c64-9aa5-6315a23122fe None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:46.004734 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4281bedb-4754-4c64-9aa5-6315a23122fe None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:46.005669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281bedb-4754-4c64-9aa5-6315a23122fe None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:46.006159 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281bedb-4754-4c64-9aa5-6315a23122fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:46.008066 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4281bedb-4754-4c64-9aa5-6315a23122fe None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:46.017066 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 444/885] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 23 16:54:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:46.023486 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4e2e6-c3ac-4556-a914-b025607f2231 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:46.131569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a4e2e6-c3ac-4556-a914-b025607f2231 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:46.166899 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f1c05-b2c0-4a7b-9ac4-683db3cef198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:46.167729 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f1c05-b2c0-4a7b-9ac4-683db3cef198 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:46.168011 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f1c05-b2c0-4a7b-9ac4-683db3cef198 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:46.168284 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f1c05-b2c0-4a7b-9ac4-683db3cef198 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:46.179274 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f1c05-b2c0-4a7b-9ac4-683db3cef198 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:46.180742 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f1c05-b2c0-4a7b-9ac4-683db3cef198 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:46.198190 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-e93f1c05-b2c0-4a7b-9ac4-683db3cef198 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:46.198190 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:46.198190 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:46.198190 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:46.198190 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:46.198190 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:46.198190 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:46.198190 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:46.198190 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:46.198190 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:46.198190 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:46.198190 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:46.198190 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:46.198190 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:46.198190 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:46.199164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:46.199164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:46.199164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:46.199164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:46.199164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:46.199164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:46.199164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:46.199164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:46.199164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:46.199164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:46.199164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:46.199164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:46.199164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:46.199164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:46.199164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:46.199164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:46.199164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:46.200057 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:46.200057 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:46.200057 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:46.200057 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:46.200057 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 442/886] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:46.207015 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281bedb-4754-4c64-9aa5-6315a23122fe None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:46.319463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4281bedb-4754-4c64-9aa5-6315a23122fe None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:46.354328 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed8811-106a-4047-894b-50a0dd0e280b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:46.355025 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed8811-106a-4047-894b-50a0dd0e280b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:46.355305 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed8811-106a-4047-894b-50a0dd0e280b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:46.355576 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed8811-106a-4047-894b-50a0dd0e280b None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:46.356292 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed8811-106a-4047-894b-50a0dd0e280b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:46.356596 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed8811-106a-4047-894b-50a0dd0e280b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:46.357964 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ed8811-106a-4047-894b-50a0dd0e280b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:46.370529 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 445/887] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:46.376259 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f1c05-b2c0-4a7b-9ac4-683db3cef198 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:46.422307 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:46.485114 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f1c05-b2c0-4a7b-9ac4-683db3cef198 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:46.521285 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d2a3d-66db-4980-9215-72cde77daf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:46.522115 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d2a3d-66db-4980-9215-72cde77daf48 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:46.522396 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d2a3d-66db-4980-9215-72cde77daf48 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:46.522666 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4d2a3d-66db-4980-9215-72cde77daf48 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:46.530645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d2a3d-66db-4980-9215-72cde77daf48 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:46.532137 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4d2a3d-66db-4980-9215-72cde77daf48 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:46.546958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-1e4d2a3d-66db-4980-9215-72cde77daf48 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 16:54:46.546958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:46.546958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:46.546958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:46.546958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:46.546958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:46.546958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:46.546958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:46.546958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:46.546958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:46.546958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:46.546958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:46.546958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:46.546958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:46.546958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:46.546958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:46.547962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:46.547962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:46.547962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:46.547962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:46.547962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:46.547962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:46.547962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:46.547962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:46.547962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:46.547962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:46.547962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:46.547962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:46.547962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:46.547962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:46.547962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:46.547962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:46.547962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:46.547962 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:46.548927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:46.548927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:46.548927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:46.548927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:46.548927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:46.548927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:46.548927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:46.548927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:46.548927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 16:54:46.548927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:46.548927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 443/888] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 23 16:54:46 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:46.555896 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed8811-106a-4047-894b-50a0dd0e280b None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:46.576024 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:46.670808 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ed8811-106a-4047-894b-50a0dd0e280b None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:46.705601 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46eaead0-3809-4e45-9658-6dc51926ddaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:46.706214 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eaead0-3809-4e45-9658-6dc51926ddaa None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:46.706499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eaead0-3809-4e45-9658-6dc51926ddaa None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:46.706767 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46eaead0-3809-4e45-9658-6dc51926ddaa None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:46.707671 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eaead0-3809-4e45-9658-6dc51926ddaa None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:46.707981 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eaead0-3809-4e45-9658-6dc51926ddaa None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:46.709460 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46eaead0-3809-4e45-9658-6dc51926ddaa None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:46.730334 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 446/889] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 23 16:54:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:46.736242 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d2a3d-66db-4980-9215-72cde77daf48 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:46.813321 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999c22d-42aa-4ecd-b65c-1fa74a9ac7fa None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:46.813669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999c22d-42aa-4ecd-b65c-1fa74a9ac7fa None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:46.813873 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4999c22d-42aa-4ecd-b65c-1fa74a9ac7fa None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:46.815712 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 447/890] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:54:46.821056 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c398d-6f40-4391-af1c-c823c4cbbd18 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:46.821336 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c398d-6f40-4391-af1c-c823c4cbbd18 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:46.821605 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814c398d-6f40-4391-af1c-c823c4cbbd18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:46.840200 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-814c398d-6f40-4391-af1c-c823c4cbbd18 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:46.841476 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-814c398d-6f40-4391-af1c-c823c4cbbd18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:46.845056 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4d2a3d-66db-4980-9215-72cde77daf48 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:46.846934 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:46.891027 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd464ad3-c634-4434-b656-f8dfae39f034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:46.891958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd464ad3-c634-4434-b656-f8dfae39f034 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:46.892077 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd464ad3-c634-4434-b656-f8dfae39f034 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:46.892330 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd464ad3-c634-4434-b656-f8dfae39f034 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:46.927230 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 448/891] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:54:46.933106 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c7ac8-0ef2-4292-a4d3-2cb1842435c2 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:46.933393 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c7ac8-0ef2-4292-a4d3-2cb1842435c2 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:46.933667 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c7ac8-0ef2-4292-a4d3-2cb1842435c2 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:46.935295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 449/892] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:54:46.940565 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d504b8-30b1-42b2-999d-37355414db14 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:46.941167 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d504b8-30b1-42b2-999d-37355414db14 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:46.941440 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03d504b8-30b1-42b2-999d-37355414db14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:46.955718 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:46.961861 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-03d504b8-30b1-42b2-999d-37355414db14 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:46.963055 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-03d504b8-30b1-42b2-999d-37355414db14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:46.965463 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd464ad3-c634-4434-b656-f8dfae39f034 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:46.966957 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd464ad3-c634-4434-b656-f8dfae39f034 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:47.034802 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 450/893] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:46 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (3 switches on core 0) May 23 16:54:47.035688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d504b8-30b1-42b2-999d-37355414db14 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:47.070163 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 444/894] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:54:46 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 16:54:47.071729 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd464ad3-c634-4434-b656-f8dfae39f034 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:47.074330 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:47.157102 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03d504b8-30b1-42b2-999d-37355414db14 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:47.180306 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd464ad3-c634-4434-b656-f8dfae39f034 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:47.193374 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce8d6b-bda1-4bdb-9ac8-79ccf1e097f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:47.194058 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce8d6b-bda1-4bdb-9ac8-79ccf1e097f8 None admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:47.194344 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce8d6b-bda1-4bdb-9ac8-79ccf1e097f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:47.194620 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ce8d6b-bda1-4bdb-9ac8-79ccf1e097f8 None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:47.195388 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce8d6b-bda1-4bdb-9ac8-79ccf1e097f8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:47.196775 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ce8d6b-bda1-4bdb-9ac8-79ccf1e097f8 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:47.199956 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:47.223385 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 451/895] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 23 16:54:47 2022] POST /identity/v3/services => generated 241 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:54:47.224179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce8d6b-bda1-4bdb-9ac8-79ccf1e097f8 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:47.228482 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ca378-8f75-47e9-aac9-d549915d10b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:47.229888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ca378-8f75-47e9-aac9-d549915d10b9 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:47.230400 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ca378-8f75-47e9-aac9-d549915d10b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:47.230883 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ca378-8f75-47e9-aac9-d549915d10b9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:47.231968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ca378-8f75-47e9-aac9-d549915d10b9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:47.233737 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ca378-8f75-47e9-aac9-d549915d10b9 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:47.250513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-de7ca378-8f75-47e9-aac9-d549915d10b9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 16:54:47.250513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:47.250513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:54:47.250513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:54:47.250513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:54:47.250513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:54:47.250513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:54:47.250513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:47.250513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 16:54:47.250513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:47.250513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:54:47.250513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:47.250513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:47.250513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:47.250513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:47.250513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:47.250513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:47.251860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:47.251860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:47.251860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:47.251860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:47.251860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:47.251860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:47.251860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:54:47.251860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:54:47.251860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:54:47.251860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:54:47.251860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:47.251860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:47.251860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:47.251860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:47.251860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:47.251860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:47.252968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:47.252968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:47.252968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:47.252968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:47.252968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:47.252968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:47.252968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:47.252968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:47.252968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:47.252968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:47.252968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:54:47.252968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:47.252968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 16:54:47.252968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:47.256399 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 445/896] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 23 16:54:47 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:47.262982 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ca378-8f75-47e9-aac9-d549915d10b9 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:47.338939 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:47.356777 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ce8d6b-bda1-4bdb-9ac8-79ccf1e097f8 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:47.386817 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ca378-8f75-47e9-aac9-d549915d10b9 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:47.394770 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cfd3ad-c05b-414e-8d07-36490bbeafdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:47.396046 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfd3ad-c05b-414e-8d07-36490bbeafdf None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:47.396046 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfd3ad-c05b-414e-8d07-36490bbeafdf None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:47.396363 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cfd3ad-c05b-414e-8d07-36490bbeafdf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:47.408712 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cfd3ad-c05b-414e-8d07-36490bbeafdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:47.410213 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cfd3ad-c05b-414e-8d07-36490bbeafdf None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:47.429301 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172172d6-d6f4-4e7e-875b-7446fecde7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:47.429521 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-14cfd3ad-c05b-414e-8d07-36490bbeafdf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:47.429521 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:47.429521 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:47.429521 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:47.429521 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:47.429521 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:47.429521 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:47.429521 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:47.429521 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:47.429521 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:47.429521 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:47.429521 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:47.429521 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:47.429521 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:47.429521 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:47.430337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:47.430337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:47.430337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:47.430337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:47.430337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:47.430337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:47.430337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:47.430337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:47.430337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:47.430337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:47.430337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:47.430337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:47.430337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:47.430337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:47.430337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:47.430337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:47.430337 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:47.431673 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:47.431673 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:47.431673 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:47.431673 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:47.432150 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172172d6-d6f4-4e7e-875b-7446fecde7af None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:47.432150 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172172d6-d6f4-4e7e-875b-7446fecde7af None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:47.432150 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172172d6-d6f4-4e7e-875b-7446fecde7af None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:47.432350 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 452/897] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:47.432418 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172172d6-d6f4-4e7e-875b-7446fecde7af None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:47.432418 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172172d6-d6f4-4e7e-875b-7446fecde7af None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:47.432756 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba3cbf-e406-4bf6-b61e-1fe69dc80151 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:47.432923 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172172d6-d6f4-4e7e-875b-7446fecde7af None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:47.433012 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba3cbf-e406-4bf6-b61e-1fe69dc80151 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:47.433810 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caba3cbf-e406-4bf6-b61e-1fe69dc80151 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:47.434620 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 453/898] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:47.437903 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba3cbf-e406-4bf6-b61e-1fe69dc80151 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:47.438367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 446/899] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 23 16:54:47 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:47.441310 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c5019-e9bf-45cc-ba89-c2cf9d175df6 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:47.441603 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c5019-e9bf-45cc-ba89-c2cf9d175df6 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:47.441885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9c5019-e9bf-45cc-ba89-c2cf9d175df6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:47.459089 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-dd9c5019-e9bf-45cc-ba89-c2cf9d175df6 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:47.460281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-dd9c5019-e9bf-45cc-ba89-c2cf9d175df6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:47.500719 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:47.531227 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 447/900] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:47 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 16:54:47.531921 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c5019-e9bf-45cc-ba89-c2cf9d175df6 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:47.563659 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caba3cbf-e406-4bf6-b61e-1fe69dc80151 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:47.600963 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55c298-cb57-4f2a-b50f-ff4a67ce2ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:47.601661 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55c298-cb57-4f2a-b50f-ff4a67ce2ebd None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:47.602015 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55c298-cb57-4f2a-b50f-ff4a67ce2ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:47.602300 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f55c298-cb57-4f2a-b50f-ff4a67ce2ebd None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:47.603060 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55c298-cb57-4f2a-b50f-ff4a67ce2ebd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:47.603373 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55c298-cb57-4f2a-b50f-ff4a67ce2ebd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:47.604933 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f55c298-cb57-4f2a-b50f-ff4a67ce2ebd None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:47.605391 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:47.621338 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 454/901] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:47.621338 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55c298-cb57-4f2a-b50f-ff4a67ce2ebd None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:47.660340 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9c5019-e9bf-45cc-ba89-c2cf9d175df6 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:47.695337 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b1057-8627-482f-a79e-067eb4da20b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:47.696005 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b1057-8627-482f-a79e-067eb4da20b5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:47.696283 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b1057-8627-482f-a79e-067eb4da20b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:47.696556 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633b1057-8627-482f-a79e-067eb4da20b5 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:47.697265 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633b1057-8627-482f-a79e-067eb4da20b5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:47.697570 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633b1057-8627-482f-a79e-067eb4da20b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:47.698967 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633b1057-8627-482f-a79e-067eb4da20b5 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:47.708824 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 448/902] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 23 16:54:47 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:47.708824 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b1057-8627-482f-a79e-067eb4da20b5 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:47.736268 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f55c298-cb57-4f2a-b50f-ff4a67ce2ebd None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:47.748127 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:47.775775 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f877e2-4da6-4e7e-8fbf-288fc1a0fbb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:47.776601 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f877e2-4da6-4e7e-8fbf-288fc1a0fbb8 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:47.776885 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f877e2-4da6-4e7e-8fbf-288fc1a0fbb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:47.777152 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f877e2-4da6-4e7e-8fbf-288fc1a0fbb8 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:47.777812 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f877e2-4da6-4e7e-8fbf-288fc1a0fbb8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:47.779213 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67f877e2-4da6-4e7e-8fbf-288fc1a0fbb8 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:47.791270 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-67f877e2-4da6-4e7e-8fbf-288fc1a0fbb8 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 16:54:47.791270 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:47.791270 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:54:47.791270 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:54:47.791270 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:54:47.791270 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:54:47.791270 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:54:47.791270 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:47.791270 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 16:54:47.791270 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:47.791270 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:54:47.791270 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:47.791270 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:47.791270 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:47.791270 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:47.791270 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:47.792266 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:47.792266 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:47.792266 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:47.792266 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:47.792266 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:47.792266 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:47.792266 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:47.792266 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:54:47.792266 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:54:47.792266 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:54:47.792266 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:54:47.792266 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:47.792266 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:47.792266 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:47.792266 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:47.792266 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:47.793099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:47.793099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:47.793099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:47.793099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:47.793099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:47.793099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:47.793099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:47.793099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:47.793099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:47.793099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:47.793099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:47.793099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:54:47.793099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:47.793099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 16:54:47.793099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:47.793099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 455/903] 10.222.0.13 () {56 vars in 1132 bytes} [Mon May 23 16:54:47 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:47.794712 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f877e2-4da6-4e7e-8fbf-288fc1a0fbb8 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:47.818859 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633b1057-8627-482f-a79e-067eb4da20b5 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:47.839605 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:47.854100 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bddd520-6750-4ab0-911c-2e3ee13c6844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:47.854900 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddd520-6750-4ab0-911c-2e3ee13c6844 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:47.855439 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddd520-6750-4ab0-911c-2e3ee13c6844 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:47.855558 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bddd520-6750-4ab0-911c-2e3ee13c6844 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:47.862269 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bddd520-6750-4ab0-911c-2e3ee13c6844 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:47.863730 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bddd520-6750-4ab0-911c-2e3ee13c6844 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:47.877181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-8bddd520-6750-4ab0-911c-2e3ee13c6844 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:47.877181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:47.877181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:47.877181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:47.877181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:47.877181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:47.877181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:47.877181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:47.877181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:47.877181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:47.877181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:47.877181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:47.877181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:47.877181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:47.877181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:47.878473 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:47.878473 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:47.878473 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:47.878473 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:47.878473 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:47.878473 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:47.878473 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:47.878473 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:47.878473 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:47.878473 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:47.878473 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:47.878473 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:47.878473 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:47.878473 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:47.878473 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:47.878473 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:47.878473 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:47.879441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:47.879441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:47.879441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:47.879441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:47.879441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:47.879441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:47.879441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:47.879441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:47.879441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 449/904] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:47.879954 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bddd520-6750-4ab0-911c-2e3ee13c6844 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:47.910981 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f877e2-4da6-4e7e-8fbf-288fc1a0fbb8 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:47.949725 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85044fbe-0038-4010-be1c-2c54af8016e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:47.950399 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85044fbe-0038-4010-be1c-2c54af8016e0 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:47.950772 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85044fbe-0038-4010-be1c-2c54af8016e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:47.951145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85044fbe-0038-4010-be1c-2c54af8016e0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:47.952096 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85044fbe-0038-4010-be1c-2c54af8016e0 None admin] RBAC: Adding query filter params (interface=public, service_id=25728fd4a5f348898d479acfc259a00f, region_id=RegionOne) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:47.952387 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85044fbe-0038-4010-be1c-2c54af8016e0 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:47.953769 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85044fbe-0038-4010-be1c-2c54af8016e0 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:47.964560 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 456/905] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 23 16:54:47 2022] GET /identity/v3/endpoints?service_id=25728fd4a5f348898d479acfc259a00f&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:47.964560 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85044fbe-0038-4010-be1c-2c54af8016e0 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:47.990544 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bddd520-6750-4ab0-911c-2e3ee13c6844 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:48.002134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:48.026571 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258b2e2-f8bd-47d4-89bf-47947f660dce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:48.027386 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258b2e2-f8bd-47d4-89bf-47947f660dce None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:48.027669 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258b2e2-f8bd-47d4-89bf-47947f660dce None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:48.027944 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8258b2e2-f8bd-47d4-89bf-47947f660dce None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:48.028682 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258b2e2-f8bd-47d4-89bf-47947f660dce None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:48.028989 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258b2e2-f8bd-47d4-89bf-47947f660dce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:48.030374 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8258b2e2-f8bd-47d4-89bf-47947f660dce None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:48.035758 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 450/906] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 23 16:54:47 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:48.036480 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258b2e2-f8bd-47d4-89bf-47947f660dce None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:48.084765 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:48.090743 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85044fbe-0038-4010-be1c-2c54af8016e0 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:48.128234 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc1a19-004f-427b-ba20-aac3ff3b355b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:48.128896 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc1a19-004f-427b-ba20-aac3ff3b355b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:48.129231 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc1a19-004f-427b-ba20-aac3ff3b355b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:48.129511 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fc1a19-004f-427b-ba20-aac3ff3b355b None admin] PATH_INFO: `/v3/projects` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:48.130442 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc1a19-004f-427b-ba20-aac3ff3b355b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:48.130663 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc1a19-004f-427b-ba20-aac3ff3b355b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:48.132144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fc1a19-004f-427b-ba20-aac3ff3b355b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:48.143268 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8258b2e2-f8bd-47d4-89bf-47947f660dce None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:48.144721 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 457/907] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:48.149680 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc1a19-004f-427b-ba20-aac3ff3b355b None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:48.183337 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc8429-cf2d-4edf-82a5-84f3e331094f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:48.184007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc8429-cf2d-4edf-82a5-84f3e331094f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:48.184289 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc8429-cf2d-4edf-82a5-84f3e331094f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:48.184568 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cc8429-cf2d-4edf-82a5-84f3e331094f None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:48.187554 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc8429-cf2d-4edf-82a5-84f3e331094f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:48.187554 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cc8429-cf2d-4edf-82a5-84f3e331094f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:48.202941 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 451/908] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 23 16:54:48 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:54:48.202941 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc8429-cf2d-4edf-82a5-84f3e331094f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:48.272103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:48.281940 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fc1a19-004f-427b-ba20-aac3ff3b355b None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:48.323496 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4fd19-b59d-4003-8f3f-9a51d9ed2d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:48.324338 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4fd19-b59d-4003-8f3f-9a51d9ed2d1c None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:48.324753 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4fd19-b59d-4003-8f3f-9a51d9ed2d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:48.325302 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4fd19-b59d-4003-8f3f-9a51d9ed2d1c None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:48.326129 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4fd19-b59d-4003-8f3f-9a51d9ed2d1c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:48.326542 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4fd19-b59d-4003-8f3f-9a51d9ed2d1c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:48.328099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4fd19-b59d-4003-8f3f-9a51d9ed2d1c None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:48.337427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cc8429-cf2d-4edf-82a5-84f3e331094f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:48.339440 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 458/909] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 23 16:54:48 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:48.341035 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4fd19-b59d-4003-8f3f-9a51d9ed2d1c None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:48.352235 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:48.391603 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0dd73e-f590-4739-bf16-715a8b4e9a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:48.392280 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0dd73e-f590-4739-bf16-715a8b4e9a19 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:48.394082 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0dd73e-f590-4739-bf16-715a8b4e9a19 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:48.394082 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0dd73e-f590-4739-bf16-715a8b4e9a19 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:48.394410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0dd73e-f590-4739-bf16-715a8b4e9a19 None admin] RBAC: Adding query filter params (role.id=7141f022bbca499c80cc11cca858e064, scope.project.id=0595667fb2e04a3ca4716f669d6698b0, user.id=4b2d638e6bd240ccbe45228352900bdb) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:48.396783 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0dd73e-f590-4739-bf16-715a8b4e9a19 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:48.397039 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0dd73e-f590-4739-bf16-715a8b4e9a19 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:48.404935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 452/910] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:54:48 2022] GET /identity/v3/role_assignments?user.id=4b2d638e6bd240ccbe45228352900bdb&scope.project.id=0595667fb2e04a3ca4716f669d6698b0&role.id=7141f022bbca499c80cc11cca858e064 => generated 259 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:48.464531 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4fd19-b59d-4003-8f3f-9a51d9ed2d1c None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:48.503743 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6716f4d8-e389-4016-ac9a-d4bac7579dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:48.504571 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716f4d8-e389-4016-ac9a-d4bac7579dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:48.504915 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716f4d8-e389-4016-ac9a-d4bac7579dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:48.505255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6716f4d8-e389-4016-ac9a-d4bac7579dd2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:48.506162 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6716f4d8-e389-4016-ac9a-d4bac7579dd2 None admin] RBAC: Adding query filter params (interface=public, service_id=4c3e1670772e43038d90086de5eaa2df, region_id=RegionOne) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:48.506543 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6716f4d8-e389-4016-ac9a-d4bac7579dd2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:48.508359 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6716f4d8-e389-4016-ac9a-d4bac7579dd2 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:48.514715 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 459/911] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 23 16:54:48 2022] GET /identity/v3/endpoints?service_id=4c3e1670772e43038d90086de5eaa2df&interface=public®ion_id=RegionOne => generated 194 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:48.520675 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0dd73e-f590-4739-bf16-715a8b4e9a19 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:48.632335 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0dd73e-f590-4739-bf16-715a8b4e9a19 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:48.668334 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f57b6aa-3ff3-4408-b65d-e6442f30d76a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:48.669010 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57b6aa-3ff3-4408-b65d-e6442f30d76a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:48.669292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57b6aa-3ff3-4408-b65d-e6442f30d76a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:48.669570 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f57b6aa-3ff3-4408-b65d-e6442f30d76a None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:48.670252 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57b6aa-3ff3-4408-b65d-e6442f30d76a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:48.671657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f57b6aa-3ff3-4408-b65d-e6442f30d76a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:48.678524 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 453/912] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 23 16:54:48 2022] GET /identity/v3/services => generated 1497 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:54:49.489233 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b157b6d-ad84-41f4-ac64-a5fc3f3c43d3 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:49.490257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b157b6d-ad84-41f4-ac64-a5fc3f3c43d3 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:49.490761 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b157b6d-ad84-41f4-ac64-a5fc3f3c43d3 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:49.492582 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 460/913] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:49.498456 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e6107-f277-49fa-8412-b287252b51f4 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:49.499308 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e6107-f277-49fa-8412-b287252b51f4 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:49.499807 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e6107-f277-49fa-8412-b287252b51f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:49.517943 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-fb0e6107-f277-49fa-8412-b287252b51f4 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:49.519172 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-fb0e6107-f277-49fa-8412-b287252b51f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:49.549615 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:49.592370 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 454/914] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:49 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:54:49.615203 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b157b6d-ad84-41f4-ac64-a5fc3f3c43d3 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:49.642045 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:49.721114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b157b6d-ad84-41f4-ac64-a5fc3f3c43d3 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:49.758045 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8583d21-fdac-4340-a5de-4e19309e9a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:49.759261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8583d21-fdac-4340-a5de-4e19309e9a99 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:49.759760 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8583d21-fdac-4340-a5de-4e19309e9a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:49.760244 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8583d21-fdac-4340-a5de-4e19309e9a99 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:49.761094 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8583d21-fdac-4340-a5de-4e19309e9a99 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:49.762682 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8583d21-fdac-4340-a5de-4e19309e9a99 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:49.775386 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-d8583d21-fdac-4340-a5de-4e19309e9a99 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 16:54:49.775386 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:49.775386 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:54:49.775386 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:54:49.775386 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:54:49.775386 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:54:49.775386 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:54:49.775386 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:49.775386 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 16:54:49.775386 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:49.775386 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:54:49.775386 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:49.775386 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:49.775386 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:49.775386 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:49.775386 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:49.775386 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:49.776412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:49.776412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:49.776412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:49.776412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:49.776412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:49.776412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:49.776412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:54:49.776412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:54:49.776412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:54:49.776412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:54:49.776412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:49.776412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:49.776412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:49.776412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:49.776412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:49.776412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:49.777262 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:49.777262 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:49.777262 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:49.777262 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:49.777262 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:49.777262 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:49.777262 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:49.777262 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:49.777262 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:49.777262 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:49.777262 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:54:49.777262 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:49.777262 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 16:54:49.777262 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:49.780248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 461/915] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 23 16:54:49 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:49.788089 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e6107-f277-49fa-8412-b287252b51f4 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:49.896792 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e6107-f277-49fa-8412-b287252b51f4 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:49.932895 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aec6ca-da9a-449f-b7a2-0b6c763cf5aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:49.933568 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aec6ca-da9a-449f-b7a2-0b6c763cf5aa None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:49.933850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aec6ca-da9a-449f-b7a2-0b6c763cf5aa None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:49.934123 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7aec6ca-da9a-449f-b7a2-0b6c763cf5aa None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:49.934827 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aec6ca-da9a-449f-b7a2-0b6c763cf5aa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:49.935240 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aec6ca-da9a-449f-b7a2-0b6c763cf5aa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:49.936623 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7aec6ca-da9a-449f-b7a2-0b6c763cf5aa None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:49.942165 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 455/916] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 23 16:54:49 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:49.947799 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8583d21-fdac-4340-a5de-4e19309e9a99 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:50.039355 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:50.052092 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8583d21-fdac-4340-a5de-4e19309e9a99 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:50.087471 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c77ee-a629-4a8b-aa82-f4c717fe6538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:50.088427 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c77ee-a629-4a8b-aa82-f4c717fe6538 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:50.088856 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c77ee-a629-4a8b-aa82-f4c717fe6538 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:50.089278 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128c77ee-a629-4a8b-aa82-f4c717fe6538 None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:50.090139 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c77ee-a629-4a8b-aa82-f4c717fe6538 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:50.090607 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c77ee-a629-4a8b-aa82-f4c717fe6538 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:50.092152 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-128c77ee-a629-4a8b-aa82-f4c717fe6538 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:50.097869 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 462/917] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 23 16:54:49 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:50.103722 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aec6ca-da9a-449f-b7a2-0b6c763cf5aa None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:50.136179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:50.208835 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7aec6ca-da9a-449f-b7a2-0b6c763cf5aa None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:50.244420 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3adbd345-ece8-40a1-839a-6c2d9949a16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:50.245094 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbd345-ece8-40a1-839a-6c2d9949a16e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:50.245377 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbd345-ece8-40a1-839a-6c2d9949a16e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:50.245644 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3adbd345-ece8-40a1-839a-6c2d9949a16e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:50.246441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adbd345-ece8-40a1-839a-6c2d9949a16e None admin] RBAC: Adding query filter params (interface=public, service_id=fe5a4b5bcc57490ab5f97e67634ba850, region_id=RegionOne) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:50.246756 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adbd345-ece8-40a1-839a-6c2d9949a16e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:50.248173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3adbd345-ece8-40a1-839a-6c2d9949a16e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:50.254748 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 456/918] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 23 16:54:50 2022] GET /identity/v3/endpoints?service_id=fe5a4b5bcc57490ab5f97e67634ba850&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:50.260968 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c77ee-a629-4a8b-aa82-f4c717fe6538 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:50.366031 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-128c77ee-a629-4a8b-aa82-f4c717fe6538 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:50.401876 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e617fc64-0d18-4947-9ea8-5db24a7cb3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:50.402939 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617fc64-0d18-4947-9ea8-5db24a7cb3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:50.403428 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617fc64-0d18-4947-9ea8-5db24a7cb3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:50.403922 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e617fc64-0d18-4947-9ea8-5db24a7cb3a8 None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:50.404805 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e617fc64-0d18-4947-9ea8-5db24a7cb3a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:50.406403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e617fc64-0d18-4947-9ea8-5db24a7cb3a8 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:50.413073 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 463/919] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 23 16:54:50 2022] GET /identity/v3/services => generated 1497 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:54:50.461130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20231ece-42a2-4e4d-9ab8-07982e9d266f None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:50.461445 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20231ece-42a2-4e4d-9ab8-07982e9d266f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:50.461683 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20231ece-42a2-4e4d-9ab8-07982e9d266f None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:50.463331 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 457/920] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:50.468777 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1802c-2e33-4a15-80db-f4eff84c2198 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:50.469077 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1802c-2e33-4a15-80db-f4eff84c2198 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:50.469361 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e1802c-2e33-4a15-80db-f4eff84c2198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:50.487879 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-26e1802c-2e33-4a15-80db-f4eff84c2198 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:50.489108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-26e1802c-2e33-4a15-80db-f4eff84c2198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:50.502000 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:50.570684 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 464/921] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:54:50.592911 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20231ece-42a2-4e4d-9ab8-07982e9d266f None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:50.692682 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:50.701393 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20231ece-42a2-4e4d-9ab8-07982e9d266f None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:50.736490 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160298d9-4705-4373-8ae9-176aa8274f9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:50.737644 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160298d9-4705-4373-8ae9-176aa8274f9a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:50.738095 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160298d9-4705-4373-8ae9-176aa8274f9a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:50.738554 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160298d9-4705-4373-8ae9-176aa8274f9a None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:50.739420 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160298d9-4705-4373-8ae9-176aa8274f9a None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:50.741050 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160298d9-4705-4373-8ae9-176aa8274f9a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:50.742862 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fb533-d196-4ab4-a62e-444a4a4de438 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:50.743283 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fb533-d196-4ab4-a62e-444a4a4de438 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:50.743664 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77fb533-d196-4ab4-a62e-444a4a4de438 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:50.745481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 465/922] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:50.750790 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46cd8b-3805-4e03-8974-4f8f4f20b6ac None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:50.751222 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46cd8b-3805-4e03-8974-4f8f4f20b6ac None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:50.751600 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f46cd8b-3805-4e03-8974-4f8f4f20b6ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:50.755200 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-160298d9-4705-4373-8ae9-176aa8274f9a None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 16:54:50.755200 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:50.755200 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:54:50.755200 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:54:50.755200 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:54:50.755200 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:54:50.755200 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:54:50.755200 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:50.755200 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 16:54:50.755200 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:50.755200 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:54:50.755200 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:50.755200 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:50.755200 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:50.755200 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:50.755200 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:50.755200 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:50.756383 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:50.756383 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:50.756383 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:50.756383 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:50.756383 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:50.756383 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:50.756383 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:54:50.756383 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:54:50.756383 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:54:50.756383 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:54:50.756383 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:50.756383 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:50.756383 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:50.756383 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:50.756383 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:50.756383 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:50.757426 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:50.757426 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:50.757426 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:50.757426 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:50.757426 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:50.757426 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:50.757426 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:50.757426 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:50.757426 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:50.757426 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:50.757426 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:54:50.757426 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:50.757426 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 16:54:50.757426 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:50.760598 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 458/923] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 23 16:54:50 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:50.767154 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160298d9-4705-4373-8ae9-176aa8274f9a None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:50.774245 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-0f46cd8b-3805-4e03-8974-4f8f4f20b6ac None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:50.775652 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-0f46cd8b-3805-4e03-8974-4f8f4f20b6ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:50.855218 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 466/924] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:50 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 16:54:50.868546 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46cd8b-3805-4e03-8974-4f8f4f20b6ac None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:50.878373 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160298d9-4705-4373-8ae9-176aa8274f9a None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:50.914720 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf6736-b96a-4a90-9707-8de896272e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:50.915863 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf6736-b96a-4a90-9707-8de896272e6c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:50.916332 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf6736-b96a-4a90-9707-8de896272e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:50.916831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bf6736-b96a-4a90-9707-8de896272e6c None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:50.917707 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf6736-b96a-4a90-9707-8de896272e6c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:50.918141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf6736-b96a-4a90-9707-8de896272e6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:50.919804 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bf6736-b96a-4a90-9707-8de896272e6c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:50.925910 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 459/925] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 23 16:54:50 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:50.931104 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf6736-b96a-4a90-9707-8de896272e6c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:50.948108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:50.959883 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:50.972584 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f46cd8b-3805-4e03-8974-4f8f4f20b6ac None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:51.008174 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019b2526-4277-465b-a4e7-ed901d9eb3c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:51.009002 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b2526-4277-465b-a4e7-ed901d9eb3c5 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:51.009286 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b2526-4277-465b-a4e7-ed901d9eb3c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:51.009560 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019b2526-4277-465b-a4e7-ed901d9eb3c5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:51.020117 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019b2526-4277-465b-a4e7-ed901d9eb3c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:51.028334 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-019b2526-4277-465b-a4e7-ed901d9eb3c5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 16:54:51.028334 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:51.028334 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:51.028334 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:51.028334 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:51.028334 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:51.028334 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:51.028334 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:51.028334 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:51.028334 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:51.028334 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:51.028334 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:51.028334 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:51.028334 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:51.028334 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:51.028334 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:51.029250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:51.029250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:51.029250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:51.029250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:51.029250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:51.029250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:51.029250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:51.029250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:51.029250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:51.029250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:51.029250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:51.029250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:51.029250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:51.029250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:51.029250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:51.029250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:51.030096 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:51.030096 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:51.030096 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:51.030096 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:51.030096 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:51.030096 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:51.030096 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:51.030096 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:51.030096 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:51.030096 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:51.030096 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 16:54:51.030096 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:51.030096 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 467/926] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 23 16:54:50 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:54:51.031549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06af40-bd6f-4c60-908e-ff72e4883f33 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:51.031549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06af40-bd6f-4c60-908e-ff72e4883f33 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:51.031549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f06af40-bd6f-4c60-908e-ff72e4883f33 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:51.032838 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 468/927] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:51.035786 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06af40-bd6f-4c60-908e-ff72e4883f33 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:51.051695 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bf6736-b96a-4a90-9707-8de896272e6c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:51.088407 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ff93a-ce9b-4d1d-b419-b1eea097c8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:51.089084 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ff93a-ce9b-4d1d-b419-b1eea097c8b1 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:51.089369 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ff93a-ce9b-4d1d-b419-b1eea097c8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:51.089648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ff93a-ce9b-4d1d-b419-b1eea097c8b1 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:51.090359 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ff93a-ce9b-4d1d-b419-b1eea097c8b1 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:51.090672 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ff93a-ce9b-4d1d-b419-b1eea097c8b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:51.092103 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ff93a-ce9b-4d1d-b419-b1eea097c8b1 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:51.097080 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 460/928] 10.222.0.13 () {56 vars in 1114 bytes} [Mon May 23 16:54:50 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:51.098349 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e62a4d-ebe5-4090-a4ad-456c23488c6a None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:51.098641 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e62a4d-ebe5-4090-a4ad-456c23488c6a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:51.098935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e62a4d-ebe5-4090-a4ad-456c23488c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:51.115401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-70e62a4d-ebe5-4090-a4ad-456c23488c6a None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:51.116571 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-70e62a4d-ebe5-4090-a4ad-456c23488c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:51.156583 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f06af40-bd6f-4c60-908e-ff72e4883f33 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:51.189248 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 461/929] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:51 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:54:51.190209 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e62a4d-ebe5-4090-a4ad-456c23488c6a None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:51.202522 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50f115-5b3f-4b89-a63f-012e51220e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:51.203502 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50f115-5b3f-4b89-a63f-012e51220e84 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:51.203990 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50f115-5b3f-4b89-a63f-012e51220e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:51.204466 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50f115-5b3f-4b89-a63f-012e51220e84 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:51.205868 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50f115-5b3f-4b89-a63f-012e51220e84 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:51.206595 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50f115-5b3f-4b89-a63f-012e51220e84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:51.208868 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50f115-5b3f-4b89-a63f-012e51220e84 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:51.212406 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:51.216364 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:51.221497 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 469/930] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 23 16:54:51 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:51.222446 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50f115-5b3f-4b89-a63f-012e51220e84 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:51.307411 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e62a4d-ebe5-4090-a4ad-456c23488c6a None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:51.339078 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50f115-5b3f-4b89-a63f-012e51220e84 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:51.343545 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff3c6e3-3577-46a9-b5cf-d257beb0a8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:51.344218 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3c6e3-3577-46a9-b5cf-d257beb0a8d0 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:51.344500 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3c6e3-3577-46a9-b5cf-d257beb0a8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:51.344777 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff3c6e3-3577-46a9-b5cf-d257beb0a8d0 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:51.345525 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff3c6e3-3577-46a9-b5cf-d257beb0a8d0 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:51.346939 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff3c6e3-3577-46a9-b5cf-d257beb0a8d0 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:51.367241 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 462/931] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 23 16:54:51 2022] POST /identity/v3/endpoints => generated 346 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:54:51.368440 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff3c6e3-3577-46a9-b5cf-d257beb0a8d0 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:51.393688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848131a-e946-499d-a0b4-db5f900722a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:51.394600 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848131a-e946-499d-a0b4-db5f900722a9 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:51.394890 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848131a-e946-499d-a0b4-db5f900722a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:51.395197 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2848131a-e946-499d-a0b4-db5f900722a9 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:51.395896 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848131a-e946-499d-a0b4-db5f900722a9 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:51.397431 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2848131a-e946-499d-a0b4-db5f900722a9 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:51.416367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-2848131a-e946-499d-a0b4-db5f900722a9 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 16:54:51.416367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:51.416367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:54:51.416367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:54:51.416367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:54:51.416367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:54:51.416367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:54:51.416367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:51.416367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 16:54:51.416367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:51.416367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:54:51.416367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:51.416367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:51.416367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:51.416367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:51.416367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:51.417452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:51.417452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:51.417452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:51.417452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:51.417452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:51.417452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:51.417452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:51.417452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:54:51.417452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:54:51.417452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:54:51.417452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:54:51.417452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:51.417452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:51.417452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:51.417452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:51.417452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:51.418319 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:51.418319 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:51.418319 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:51.418319 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:51.418319 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:51.418319 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:51.418319 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:51.418319 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:51.418319 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:51.418319 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:51.418319 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:51.418319 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:54:51.418319 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:51.418319 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 16:54:51.418319 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:51.423356 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 470/932] 10.222.0.13 () {56 vars in 1132 bytes} [Mon May 23 16:54:51 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:51.425896 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848131a-e946-499d-a0b4-db5f900722a9 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:51.474717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:51.517401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff3c6e3-3577-46a9-b5cf-d257beb0a8d0 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:51.551296 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:51.559123 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c41d88-4be8-4b6b-a75b-779a4ac8eb74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:51.560177 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c41d88-4be8-4b6b-a75b-779a4ac8eb74 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:51.560637 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c41d88-4be8-4b6b-a75b-779a4ac8eb74 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:51.561096 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c41d88-4be8-4b6b-a75b-779a4ac8eb74 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:51.574064 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c41d88-4be8-4b6b-a75b-779a4ac8eb74 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:51.575859 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c41d88-4be8-4b6b-a75b-779a4ac8eb74 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:51.578656 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2848131a-e946-499d-a0b4-db5f900722a9 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:51.594831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-79c41d88-4be8-4b6b-a75b-779a4ac8eb74 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:51.594831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:51.594831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:51.594831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:51.594831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:51.594831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:51.594831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:51.594831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:51.594831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:51.594831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:51.594831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:51.594831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:51.594831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:51.594831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:51.594831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:51.596185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:51.596185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:51.596185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:51.596185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:51.596185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:51.596185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:51.596185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:51.596185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:51.596185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:51.596185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:51.596185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:51.596185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:51.596185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:51.596185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:51.596185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:51.596185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:51.596185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:51.597071 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:51.597071 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:51.597071 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:51.597071 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:51.597071 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 463/933] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:51.601925 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c41d88-4be8-4b6b-a75b-779a4ac8eb74 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:51.625395 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6601f2-c8ae-46ee-a964-f8172dd1ae1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:51.626295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6601f2-c8ae-46ee-a964-f8172dd1ae1e None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:51.627573 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6601f2-c8ae-46ee-a964-f8172dd1ae1e None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:51.627573 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc6601f2-c8ae-46ee-a964-f8172dd1ae1e None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:51.628036 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6601f2-c8ae-46ee-a964-f8172dd1ae1e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:51.628447 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6601f2-c8ae-46ee-a964-f8172dd1ae1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:51.630016 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc6601f2-c8ae-46ee-a964-f8172dd1ae1e None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:51.636689 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 471/934] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 23 16:54:51 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:51.642750 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6601f2-c8ae-46ee-a964-f8172dd1ae1e None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:51.721196 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c41d88-4be8-4b6b-a75b-779a4ac8eb74 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:51.745689 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:51.758883 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e6603-8e54-4e42-ba3f-b5573d77e3da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:51.759694 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e6603-8e54-4e42-ba3f-b5573d77e3da None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:51.759977 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e6603-8e54-4e42-ba3f-b5573d77e3da None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:51.760249 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad6e6603-8e54-4e42-ba3f-b5573d77e3da None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:51.761090 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e6603-8e54-4e42-ba3f-b5573d77e3da None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:51.761395 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e6603-8e54-4e42-ba3f-b5573d77e3da None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:51.762928 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad6e6603-8e54-4e42-ba3f-b5573d77e3da None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:51.769163 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc6601f2-c8ae-46ee-a964-f8172dd1ae1e None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:51.776489 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 464/935] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:51.784455 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e6603-8e54-4e42-ba3f-b5573d77e3da None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:51.795197 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:51.809208 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74a2d0-7a1d-4fef-b16f-0095b7ee52c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:51.809881 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74a2d0-7a1d-4fef-b16f-0095b7ee52c1 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:51.810157 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74a2d0-7a1d-4fef-b16f-0095b7ee52c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:51.810519 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e74a2d0-7a1d-4fef-b16f-0095b7ee52c1 None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:51.811277 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74a2d0-7a1d-4fef-b16f-0095b7ee52c1 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:51.811582 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74a2d0-7a1d-4fef-b16f-0095b7ee52c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:51.813075 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e74a2d0-7a1d-4fef-b16f-0095b7ee52c1 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:51.823150 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 472/936] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 23 16:54:51 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:51.824462 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74a2d0-7a1d-4fef-b16f-0095b7ee52c1 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:51.913993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad6e6603-8e54-4e42-ba3f-b5573d77e3da None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:51.928577 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e74a2d0-7a1d-4fef-b16f-0095b7ee52c1 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:51.959321 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc188b2a-1c08-4a77-815a-75d0a807f306 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:51.960647 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc188b2a-1c08-4a77-815a-75d0a807f306 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:51.961416 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc188b2a-1c08-4a77-815a-75d0a807f306 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:51.962148 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc188b2a-1c08-4a77-815a-75d0a807f306 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:51.965991 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f84e570-157f-4e90-bf61-e5637652cefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:51.966491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84e570-157f-4e90-bf61-e5637652cefa None admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:51.966767 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84e570-157f-4e90-bf61-e5637652cefa None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:51.967081 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f84e570-157f-4e90-bf61-e5637652cefa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:51.967821 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f84e570-157f-4e90-bf61-e5637652cefa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:51.969182 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f84e570-157f-4e90-bf61-e5637652cefa None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:51.972226 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc188b2a-1c08-4a77-815a-75d0a807f306 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:51.974333 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc188b2a-1c08-4a77-815a-75d0a807f306 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:51.991951 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-bc188b2a-1c08-4a77-815a-75d0a807f306 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 16:54:51.991951 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:51.991951 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:51.991951 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:51.991951 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:51.991951 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:51.991951 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:51.991951 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:51.991951 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:51.991951 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:51.991951 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:51.991951 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:51.991951 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:51.991951 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:51.991951 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:51.991951 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:51.992946 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:51.992946 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:51.992946 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:51.992946 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:51.992946 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:51.992946 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:51.992946 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:51.992946 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:51.992946 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:51.992946 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:51.992946 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:51.992946 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:51.992946 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:51.992946 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:51.992946 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:51.992946 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:51.992946 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:51.992946 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:51.993907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:51.993907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:51.993907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:51.993907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:51.993907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:51.993907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:51.993907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:51.993907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:51.993907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 16:54:51.993907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:51.996698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 465/938] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 23 16:54:51 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:54:51.996819 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 473/938] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 23 16:54:51 2022] POST /identity/v3/endpoints => generated 347 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:54:52.004513 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f84e570-157f-4e90-bf61-e5637652cefa None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:52.058612 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:52.135131 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f84e570-157f-4e90-bf61-e5637652cefa None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:52.175791 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee33e7-b91b-4ba3-a615-ddaba656f84b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:52.177423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee33e7-b91b-4ba3-a615-ddaba656f84b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:52.177423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee33e7-b91b-4ba3-a615-ddaba656f84b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:52.177423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bee33e7-b91b-4ba3-a615-ddaba656f84b None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:52.177895 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee33e7-b91b-4ba3-a615-ddaba656f84b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:52.178201 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee33e7-b91b-4ba3-a615-ddaba656f84b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:52.179674 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bee33e7-b91b-4ba3-a615-ddaba656f84b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:52.200309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 474/939] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 23 16:54:52 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:52.204475 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc188b2a-1c08-4a77-815a-75d0a807f306 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:52.253768 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:52.312068 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc188b2a-1c08-4a77-815a-75d0a807f306 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:52.348307 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d49c4a-918d-48b4-8843-15ee11024a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:52.349506 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d49c4a-918d-48b4-8843-15ee11024a65 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:52.350024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d49c4a-918d-48b4-8843-15ee11024a65 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:52.350496 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d49c4a-918d-48b4-8843-15ee11024a65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:52.361032 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d49c4a-918d-48b4-8843-15ee11024a65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:52.362838 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d49c4a-918d-48b4-8843-15ee11024a65 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:52.380910 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-72d49c4a-918d-48b4-8843-15ee11024a65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:52.380910 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:52.380910 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:52.380910 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:52.380910 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:52.380910 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:52.380910 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:52.380910 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:52.380910 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:52.380910 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:52.380910 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:52.380910 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:52.380910 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:52.380910 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:52.380910 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:52.381831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:52.381831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:52.381831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:52.381831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:52.381831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:52.381831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:52.381831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:52.381831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:52.381831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:52.381831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:52.381831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:52.381831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:52.381831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:52.381831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:52.381831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:52.381831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:52.381831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:52.383081 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:52.383081 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:52.383081 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:52.383081 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:52.385348 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 466/940] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:54:52.390998 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee33e7-b91b-4ba3-a615-ddaba656f84b None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:52.490704 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:52.497125 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bee33e7-b91b-4ba3-a615-ddaba656f84b None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:52.533011 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157855b-52d7-4217-ac04-a5b02d879399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:52.533655 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157855b-52d7-4217-ac04-a5b02d879399 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:52.533929 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157855b-52d7-4217-ac04-a5b02d879399 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:52.534200 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1157855b-52d7-4217-ac04-a5b02d879399 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:52.534933 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1157855b-52d7-4217-ac04-a5b02d879399 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:52.535241 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1157855b-52d7-4217-ac04-a5b02d879399 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:52.536601 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1157855b-52d7-4217-ac04-a5b02d879399 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:52.548558 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 475/941] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:52.554130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d49c4a-918d-48b4-8843-15ee11024a65 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:52.658960 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:52.665059 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d49c4a-918d-48b4-8843-15ee11024a65 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:52.700145 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b96bdf-8924-4bf0-88a3-b8ac4223d2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:52.700955 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b96bdf-8924-4bf0-88a3-b8ac4223d2da None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:52.701231 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b96bdf-8924-4bf0-88a3-b8ac4223d2da None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:52.701502 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b96bdf-8924-4bf0-88a3-b8ac4223d2da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:52.709552 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b96bdf-8924-4bf0-88a3-b8ac4223d2da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:52.711262 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b96bdf-8924-4bf0-88a3-b8ac4223d2da None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:52.726323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-d7b96bdf-8924-4bf0-88a3-b8ac4223d2da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:52.726323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:52.726323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:52.726323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:52.726323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:52.726323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:52.726323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:52.726323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:52.726323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:52.726323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:52.726323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:52.726323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:52.726323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:52.726323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:52.726323 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:52.727253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:52.727253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:52.727253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:52.727253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:52.727253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:52.727253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:52.727253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:52.727253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:52.727253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:52.727253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:52.727253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:52.727253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:52.727253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:52.727253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:52.727253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:52.727253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:52.727253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:52.728141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:52.728141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:52.728141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:52.728141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:52.728141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:52.728141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:52.728141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:52.728141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:52.728141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 467/942] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:52.733569 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157855b-52d7-4217-ac04-a5b02d879399 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:52.785520 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a154e27-3969-45a2-a18a-abef3d32240f None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:52.785711 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a154e27-3969-45a2-a18a-abef3d32240f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:52.785983 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a154e27-3969-45a2-a18a-abef3d32240f None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:52.787744 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 468/943] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:54:52.795290 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b5b4e-6f9f-4a16-a506-70558d7a26de None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:52.795678 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b5b4e-6f9f-4a16-a506-70558d7a26de None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:52.796048 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9b5b4e-6f9f-4a16-a506-70558d7a26de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:52.818942 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-4d9b5b4e-6f9f-4a16-a506-70558d7a26de None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:52.820304 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-4d9b5b4e-6f9f-4a16-a506-70558d7a26de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:52.851876 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1157855b-52d7-4217-ac04-a5b02d879399 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:52.889700 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcd55c-c07d-4c24-9ce4-93bebd06ede6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:52.890649 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcd55c-c07d-4c24-9ce4-93bebd06ede6 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:52.891126 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcd55c-c07d-4c24-9ce4-93bebd06ede6 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:52.891803 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadcd55c-c07d-4c24-9ce4-93bebd06ede6 None admin] PATH_INFO: `/v3/projects` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:52.893013 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadcd55c-c07d-4c24-9ce4-93bebd06ede6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:52.893549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadcd55c-c07d-4c24-9ce4-93bebd06ede6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:52.895399 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadcd55c-c07d-4c24-9ce4-93bebd06ede6 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:52.913815 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 476/945] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:52.915122 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 469/945] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:52 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:54:52.915773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcd55c-c07d-4c24-9ce4-93bebd06ede6 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:52.934277 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b5b4e-6f9f-4a16-a506-70558d7a26de None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:52.935912 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:53.026348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadcd55c-c07d-4c24-9ce4-93bebd06ede6 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:53.028697 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:53.043863 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9b5b4e-6f9f-4a16-a506-70558d7a26de None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:53.068241 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460fa45-6f59-46eb-8892-b361a48bfa14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:53.069651 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460fa45-6f59-46eb-8892-b361a48bfa14 None admin] REQUEST_METHOD: `PUT` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:53.070136 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460fa45-6f59-46eb-8892-b361a48bfa14 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:53.070682 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c460fa45-6f59-46eb-8892-b361a48bfa14 None admin] PATH_INFO: `/v3/projects/0595667fb2e04a3ca4716f669d6698b0/users/4b2d638e6bd240ccbe45228352900bdb/roles/7141f022bbca499c80cc11cca858e064` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:53.073602 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c460fa45-6f59-46eb-8892-b361a48bfa14 None admin] RBAC: Authorizing `identity:create_grant(project_id=0595667fb2e04a3ca4716f669d6698b0, user_id=4b2d638e6bd240ccbe45228352900bdb, role_id=7141f022bbca499c80cc11cca858e064)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:53.075559 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c460fa45-6f59-46eb-8892-b361a48bfa14 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:53.079622 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d9880-f9b6-4f6f-a159-84b00d8b7224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:53.080714 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d9880-f9b6-4f6f-a159-84b00d8b7224 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:53.081054 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d9880-f9b6-4f6f-a159-84b00d8b7224 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:53.081530 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8d9880-f9b6-4f6f-a159-84b00d8b7224 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:53.082471 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d9880-f9b6-4f6f-a159-84b00d8b7224 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:53.083911 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8d9880-f9b6-4f6f-a159-84b00d8b7224 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:53.097242 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-0b8d9880-f9b6-4f6f-a159-84b00d8b7224 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 16:54:53.097242 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:53.097242 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:54:53.097242 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:54:53.097242 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:54:53.097242 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:54:53.097242 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:54:53.097242 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:53.097242 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 16:54:53.097242 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:53.097242 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:54:53.097242 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:53.097242 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:53.097242 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:53.097242 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:53.097242 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:53.097242 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:53.098318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:53.098318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:53.098318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:53.098318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:53.098318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:53.098318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:53.098318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:54:53.098318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:54:53.098318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:54:53.098318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:54:53.098318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:53.098318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:53.098318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:53.098318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:53.098318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:53.098318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:53.099214 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:53.099214 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:53.099214 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:53.099214 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:53.099214 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:53.099214 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:53.099214 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:53.099214 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:53.099214 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:53.099214 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:53.099214 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:54:53.099214 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:53.099214 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 16:54:53.099214 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:53.100921 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 477/947] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:54:52 2022] PUT /identity/v3/projects/0595667fb2e04a3ca4716f669d6698b0/users/4b2d638e6bd240ccbe45228352900bdb/roles/7141f022bbca499c80cc11cca858e064 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:54:53.103225 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 470/947] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 23 16:54:52 2022] GET /identity/v3/services/object-store => generated 93 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:53.108390 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460fa45-6f59-46eb-8892-b361a48bfa14 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:53.202489 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:53.237528 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c460fa45-6f59-46eb-8892-b361a48bfa14 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:53.277460 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822bb55-d56d-420d-9f70-62b704f1fa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:53.278149 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822bb55-d56d-420d-9f70-62b704f1fa4a None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:53.278434 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822bb55-d56d-420d-9f70-62b704f1fa4a None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:53.278714 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822bb55-d56d-420d-9f70-62b704f1fa4a None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:53.279473 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822bb55-d56d-420d-9f70-62b704f1fa4a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:53.279782 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822bb55-d56d-420d-9f70-62b704f1fa4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:53.281196 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822bb55-d56d-420d-9f70-62b704f1fa4a None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:53.287521 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 478/948] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 23 16:54:53 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:53.293360 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d9880-f9b6-4f6f-a159-84b00d8b7224 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:53.409256 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8d9880-f9b6-4f6f-a159-84b00d8b7224 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:53.443508 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbf61cf-9703-4bf2-b4f8-680bffae1d21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:53.444170 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf61cf-9703-4bf2-b4f8-680bffae1d21 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:53.444446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf61cf-9703-4bf2-b4f8-680bffae1d21 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:53.444719 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbf61cf-9703-4bf2-b4f8-680bffae1d21 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:53.445414 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf61cf-9703-4bf2-b4f8-680bffae1d21 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:53.445713 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf61cf-9703-4bf2-b4f8-680bffae1d21 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:53.447117 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbf61cf-9703-4bf2-b4f8-680bffae1d21 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:53.452452 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 471/949] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 23 16:54:53 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:53.458262 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822bb55-d56d-420d-9f70-62b704f1fa4a None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:53.572192 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822bb55-d56d-420d-9f70-62b704f1fa4a None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:53.606837 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec289b9-42d6-469b-ae44-54f317a9b4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:53.607682 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec289b9-42d6-469b-ae44-54f317a9b4dc None admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:53.608064 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec289b9-42d6-469b-ae44-54f317a9b4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:53.608438 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec289b9-42d6-469b-ae44-54f317a9b4dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:53.609295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec289b9-42d6-469b-ae44-54f317a9b4dc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:53.610756 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec289b9-42d6-469b-ae44-54f317a9b4dc None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:53.619163 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:53.636379 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 479/950] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 23 16:54:53 2022] POST /identity/v3/endpoints => generated 349 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:54:53.739410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc1edd-f85e-4d38-9653-4d4311a8cb3e None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:53.739920 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc1edd-f85e-4d38-9653-4d4311a8cb3e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:53.740167 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdc1edd-f85e-4d38-9653-4d4311a8cb3e None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:53.742008 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 472/951] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:53.748049 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4393c0-7f7a-4cef-a929-b3629bbbf242 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:53.748552 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4393c0-7f7a-4cef-a929-b3629bbbf242 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:53.748628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4393c0-7f7a-4cef-a929-b3629bbbf242 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:53.767577 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-5d4393c0-7f7a-4cef-a929-b3629bbbf242 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:53.768878 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-5d4393c0-7f7a-4cef-a929-b3629bbbf242 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:53.848931 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 480/952] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:53 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:54:53.873757 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc1edd-f85e-4d38-9653-4d4311a8cb3e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:53.891409 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:53.980705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecdc1edd-f85e-4d38-9653-4d4311a8cb3e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:54.015793 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffca725-78c9-4955-8ae5-8714c9a04d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:54.016375 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffca725-78c9-4955-8ae5-8714c9a04d36 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:54.016651 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffca725-78c9-4955-8ae5-8714c9a04d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:54.016919 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffca725-78c9-4955-8ae5-8714c9a04d36 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:54.017634 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffca725-78c9-4955-8ae5-8714c9a04d36 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:54.019204 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffca725-78c9-4955-8ae5-8714c9a04d36 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:54.020570 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: WARNING keystone.server.flask.common [None req-8ffca725-78c9-4955-8ae5-8714c9a04d36 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 23 16:54:54.040552 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.sql.core [None req-8ffca725-78c9-4955-8ae5-8714c9a04d36 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 23 16:54:54.040552 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 23 16:54:54.040552 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [parameters: {'id': '8c9f00a87dbd4743932d269e9fd6105f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 23 16:54:54.040552 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80464) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 23 16:54:54.057741 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-8ffca725-78c9-4955-8ae5-8714c9a04d36 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8c9f00a87dbd4743932d269e9fd6105f.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8c9f00a87dbd4743932d269e9fd6105f. May 23 16:54:54.057741 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:54.057741 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 16:54:54.057741 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 16:54:54.057741 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 16:54:54.057741 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 16:54:54.057741 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 16:54:54.057741 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application result = self._query(query) May 23 16:54:54.057741 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 16:54:54.057741 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application conn.query(q) May 23 16:54:54.057741 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 16:54:54.057741 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 16:54:54.057741 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 16:54:54.058621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application result.read() May 23 16:54:54.058621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 16:54:54.058621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 16:54:54.058621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 16:54:54.058621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 16:54:54.058621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 16:54:54.058621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 16:54:54.058621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 16:54:54.058621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 16:54:54.058621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 23 16:54:54.058621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:54.058621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 23 16:54:54.058621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:54.058621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:54.058621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 23 16:54:54.058621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 23 16:54:54.058621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 23 16:54:54.058621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return project_ref.to_dict() May 23 16:54:54.059850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 16:54:54.059850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application next(self.gen) May 23 16:54:54.059850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 23 16:54:54.059850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application yield resource May 23 16:54:54.059850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 16:54:54.059850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application next(self.gen) May 23 16:54:54.059850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 23 16:54:54.059850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self.session.rollback() May 23 16:54:54.059850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 23 16:54:54.059850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self.force_reraise() May 23 16:54:54.059850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 23 16:54:54.059850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise self.value May 23 16:54:54.059850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 23 16:54:54.059850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 23 16:54:54.059850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 23 16:54:54.059850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application session.commit() May 23 16:54:54.059850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 23 16:54:54.060876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 23 16:54:54.060876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 23 16:54:54.060876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._prepare_impl() May 23 16:54:54.060876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 23 16:54:54.060876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self.session.flush() May 23 16:54:54.060876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 23 16:54:54.060876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._flush(objects) May 23 16:54:54.060876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 23 16:54:54.060876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 23 16:54:54.060876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 23 16:54:54.060876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application compat.raise_( May 23 16:54:54.060876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 16:54:54.060876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception May 23 16:54:54.060876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 23 16:54:54.060876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application flush_context.execute() May 23 16:54:54.060876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 23 16:54:54.060876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rec.execute(self) May 23 16:54:54.061789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 23 16:54:54.061789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 23 16:54:54.061789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 23 16:54:54.061789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application _emit_insert_statements( May 23 16:54:54.061789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 23 16:54:54.061789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application c = connection._execute_20( May 23 16:54:54.061789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 23 16:54:54.061789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 23 16:54:54.061789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 23 16:54:54.061789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 23 16:54:54.061789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 23 16:54:54.061789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ret = self._execute_context( May 23 16:54:54.061789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 23 16:54:54.061789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 23 16:54:54.061789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 23 16:54:54.062592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 23 16:54:54.062592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 16:54:54.062592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception May 23 16:54:54.062592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 16:54:54.062592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 16:54:54.062592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 16:54:54.062592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 16:54:54.062592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 16:54:54.062592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application result = self._query(query) May 23 16:54:54.062592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 16:54:54.062592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application conn.query(q) May 23 16:54:54.062592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 16:54:54.062592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 16:54:54.062592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 16:54:54.062592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application result.read() May 23 16:54:54.062592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 16:54:54.062592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 16:54:54.063523 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 16:54:54.063523 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 16:54:54.063523 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 16:54:54.063523 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 16:54:54.063523 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 16:54:54.063523 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 16:54:54.063523 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 23 16:54:54.063523 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 23 16:54:54.063523 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [parameters: {'id': '8c9f00a87dbd4743932d269e9fd6105f', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 23 16:54:54.063523 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 23 16:54:54.063523 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:54.063523 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:54:54.063523 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:54.063523 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:54.063523 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 23 16:54:54.066273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 23 16:54:54.066273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 23 16:54:54.066273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 23 16:54:54.066273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 23 16:54:54.066273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:54.066273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:54:54.066273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:54.066273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:54.066273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:54.066273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:54.066273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:54.066273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:54.066273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:54.066273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:54.066273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:54.066273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:54.066273 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:54.068600 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:54.068600 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 23 16:54:54.068600 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 23 16:54:54.068600 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:54.068600 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:54.068600 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 23 16:54:54.068600 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.Conflict( May 23 16:54:54.068600 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 8c9f00a87dbd4743932d269e9fd6105f. May 23 16:54:54.068600 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:54.068600 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 473/953] 10.222.0.13 () {60 vars in 1140 bytes} [Mon May 23 16:54:53 2022] POST /identity/v3/projects => generated 262 bytes in 186 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:54.069995 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4393c0-7f7a-4cef-a929-b3629bbbf242 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:54.174568 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4393c0-7f7a-4cef-a929-b3629bbbf242 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:54.203491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:54.215987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953e6b61-5c0b-4227-92e9-78a24d0c11c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:54.217175 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e6b61-5c0b-4227-92e9-78a24d0c11c7 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:54.217691 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e6b61-5c0b-4227-92e9-78a24d0c11c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:54.218166 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953e6b61-5c0b-4227-92e9-78a24d0c11c7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:54.225718 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e6b61-5c0b-4227-92e9-78a24d0c11c7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:54.227435 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953e6b61-5c0b-4227-92e9-78a24d0c11c7 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:54.241891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-953e6b61-5c0b-4227-92e9-78a24d0c11c7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:54.241891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:54.241891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:54.241891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:54.241891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:54.241891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:54.241891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:54.241891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:54.241891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:54.241891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:54.241891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:54.241891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:54.241891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:54.241891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:54.241891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:54.242818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:54.242818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:54.242818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:54.242818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:54.242818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:54.242818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:54.242818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:54.242818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:54.242818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:54.242818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:54.242818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:54.242818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:54.242818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:54.242818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:54.242818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:54.242818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:54.242818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:54.243789 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:54.243789 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:54.243789 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:54.243789 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:54.243789 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:54.243789 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:54.243789 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:54.243789 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:54.246333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 481/954] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:54.252394 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffca725-78c9-4955-8ae5-8714c9a04d36 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:54.329380 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:54.356763 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd497e-0dd1-4875-b8bf-6db187465838 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:54.357108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd497e-0dd1-4875-b8bf-6db187465838 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:54.357319 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bd497e-0dd1-4875-b8bf-6db187465838 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:54.358948 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 482/955] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:54:54.362697 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffca725-78c9-4955-8ae5-8714c9a04d36 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:54.364224 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3dd38-0e0b-448c-b911-37c21b0f195c None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:54.364609 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3dd38-0e0b-448c-b911-37c21b0f195c None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:54.365128 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3dd38-0e0b-448c-b911-37c21b0f195c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:54.386655 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-aca3dd38-0e0b-448c-b911-37c21b0f195c None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:54.388153 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-aca3dd38-0e0b-448c-b911-37c21b0f195c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:54.400438 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4663fb-effe-4f58-8f72-4d3092482dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:54.401205 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4663fb-effe-4f58-8f72-4d3092482dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:54.401490 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4663fb-effe-4f58-8f72-4d3092482dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:54.401761 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4663fb-effe-4f58-8f72-4d3092482dd2 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:54.402695 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4663fb-effe-4f58-8f72-4d3092482dd2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:54.403036 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4663fb-effe-4f58-8f72-4d3092482dd2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:54.404834 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4663fb-effe-4f58-8f72-4d3092482dd2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:54.426924 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 474/956] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 23 16:54:54 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:54.472680 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 483/957] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:54 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 16:54:54.497650 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4663fb-effe-4f58-8f72-4d3092482dd2 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:54.608142 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4663fb-effe-4f58-8f72-4d3092482dd2 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:54.638738 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:54.645036 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04579c5e-f98a-4a26-b776-5677ed56f4c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:54.645853 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04579c5e-f98a-4a26-b776-5677ed56f4c3 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:54.646134 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04579c5e-f98a-4a26-b776-5677ed56f4c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:54.646437 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04579c5e-f98a-4a26-b776-5677ed56f4c3 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:54.647108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04579c5e-f98a-4a26-b776-5677ed56f4c3 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:54.648674 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04579c5e-f98a-4a26-b776-5677ed56f4c3 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:54.661136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-04579c5e-f98a-4a26-b776-5677ed56f4c3 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 23 16:54:54.661136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:54.661136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:54:54.661136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:54:54.661136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:54:54.661136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:54:54.661136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:54:54.661136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:54.661136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 16:54:54.661136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:54.661136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:54:54.661136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:54.661136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:54.661136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:54.661136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:54.661136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:54.661136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:54.662221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:54.662221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:54.662221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:54.662221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:54.662221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:54.662221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:54.662221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:54:54.662221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:54:54.662221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:54:54.662221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:54:54.662221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:54.662221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:54.662221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:54.662221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:54.662221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:54.662221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:54.663243 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:54.663243 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:54.663243 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:54.663243 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:54.663243 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:54.663243 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:54.663243 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:54.663243 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:54.663243 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:54.663243 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:54.663243 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:54:54.663243 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:54.663243 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 16:54:54.663243 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:54.663243 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 475/958] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:54 2022] GET /identity/v3/services/compute => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:54.671461 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3dd38-0e0b-448c-b911-37c21b0f195c None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:54.784766 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3dd38-0e0b-448c-b911-37c21b0f195c None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:54.801307 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:54.824254 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73ce8eb-6c98-45f2-ae58-1e4df8508e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:54.825266 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ce8eb-6c98-45f2-ae58-1e4df8508e32 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:54.825763 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ce8eb-6c98-45f2-ae58-1e4df8508e32 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:54.826241 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73ce8eb-6c98-45f2-ae58-1e4df8508e32 None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:54.827190 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ce8eb-6c98-45f2-ae58-1e4df8508e32 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:54.827690 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ce8eb-6c98-45f2-ae58-1e4df8508e32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:54.829294 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73ce8eb-6c98-45f2-ae58-1e4df8508e32 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:54.835834 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 484/959] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 23 16:54:54 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:54.844040 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04579c5e-f98a-4a26-b776-5677ed56f4c3 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:54.945553 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04579c5e-f98a-4a26-b776-5677ed56f4c3 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:54.983204 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a774ed5-0173-4045-9a65-16171f268cc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:54.984231 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a774ed5-0173-4045-9a65-16171f268cc0 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:54.984739 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a774ed5-0173-4045-9a65-16171f268cc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:54.985239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a774ed5-0173-4045-9a65-16171f268cc0 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:54.986158 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a774ed5-0173-4045-9a65-16171f268cc0 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:54.986725 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a774ed5-0173-4045-9a65-16171f268cc0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:54.988319 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a774ed5-0173-4045-9a65-16171f268cc0 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:54.993840 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 476/960] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 23 16:54:54 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:55.450042 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeff7f1-c3af-409b-8496-cd3b76466987 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:55.450423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeff7f1-c3af-409b-8496-cd3b76466987 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:55.450599 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adeff7f1-c3af-409b-8496-cd3b76466987 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:55.452239 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 485/961] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:55.457672 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db86cd-f713-4b2c-b38d-98a0710f8dfd None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:55.457985 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db86cd-f713-4b2c-b38d-98a0710f8dfd None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:55.458232 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93db86cd-f713-4b2c-b38d-98a0710f8dfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:55.487508 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-93db86cd-f713-4b2c-b38d-98a0710f8dfd None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:55.489117 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-93db86cd-f713-4b2c-b38d-98a0710f8dfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:55.547328 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:55.563605 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 477/962] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:54:55.586557 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeff7f1-c3af-409b-8496-cd3b76466987 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:55.695108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adeff7f1-c3af-409b-8496-cd3b76466987 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:55.733462 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e2f08-c9db-4354-baee-29da17e037b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:55.734598 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e2f08-c9db-4354-baee-29da17e037b5 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:55.735078 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e2f08-c9db-4354-baee-29da17e037b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:55.735517 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9e2f08-c9db-4354-baee-29da17e037b5 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:55.746721 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9e2f08-c9db-4354-baee-29da17e037b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:54:55.755344 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-9a9e2f08-c9db-4354-baee-29da17e037b5 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 16:54:55.755344 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:55.755344 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:55.755344 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:55.755344 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:55.755344 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:55.755344 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:55.755344 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:55.755344 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:55.755344 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:55.755344 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:55.755344 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:55.755344 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:54:55.755344 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:54:55.755344 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:54:55.755344 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:54:55.756493 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:54:55.756493 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:54:55.756493 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:54:55.756493 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:54:55.756493 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:55.756493 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:55.756493 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:55.756493 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:55.756493 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:55.756493 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:55.756493 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:55.756493 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:55.756493 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:55.756493 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:55.756493 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:55.756493 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:55.757853 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:55.757853 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:55.757853 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:55.757853 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:55.757853 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:54:55.757853 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:54:55.757853 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:54:55.757853 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:54:55.757853 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:54:55.757853 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:54:55.757853 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 16:54:55.757853 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:55.760330 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 486/963] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 23 16:54:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:55.768253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db86cd-f713-4b2c-b38d-98a0710f8dfd None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:55.877228 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93db86cd-f713-4b2c-b38d-98a0710f8dfd None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:55.912639 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b62283-956a-4449-970d-62ac905e1c84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:55.913566 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b62283-956a-4449-970d-62ac905e1c84 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:55.913995 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b62283-956a-4449-970d-62ac905e1c84 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:55.914413 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b62283-956a-4449-970d-62ac905e1c84 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:55.915336 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b62283-956a-4449-970d-62ac905e1c84 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:55.915803 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b62283-956a-4449-970d-62ac905e1c84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:55.917329 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b62283-956a-4449-970d-62ac905e1c84 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:55.926130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 478/964] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 23 16:54:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:55.934294 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e2f08-c9db-4354-baee-29da17e037b5 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:55.971373 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:55.982352 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c64487-bf2f-44fa-8b3d-397be3f3a1de None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:55.982958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c64487-bf2f-44fa-8b3d-397be3f3a1de None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:55.983517 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c64487-bf2f-44fa-8b3d-397be3f3a1de None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:55.985342 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 479/965] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:54:55.990760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c5655-4b7c-4521-a4d9-844ded9563fa None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:55.991322 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c5655-4b7c-4521-a4d9-844ded9563fa None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:55.991808 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2c5655-4b7c-4521-a4d9-844ded9563fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:56.012433 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-bb2c5655-4b7c-4521-a4d9-844ded9563fa None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:56.013968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-bb2c5655-4b7c-4521-a4d9-844ded9563fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:56.041095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9e2f08-c9db-4354-baee-29da17e037b5 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:56.064141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:56.086068 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c1161a-ac83-4b55-864d-af6be69ae4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:56.087544 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1161a-ac83-4b55-864d-af6be69ae4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:56.088244 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1161a-ac83-4b55-864d-af6be69ae4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:56.088671 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c1161a-ac83-4b55-864d-af6be69ae4e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:56.099423 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 480/966] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:55 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 16:54:56.101630 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c1161a-ac83-4b55-864d-af6be69ae4e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:56.103303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c1161a-ac83-4b55-864d-af6be69ae4e5 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:56.115753 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2c5655-4b7c-4521-a4d9-844ded9563fa None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:56.123018 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-40c1161a-ac83-4b55-864d-af6be69ae4e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:56.123018 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:56.123018 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:56.123018 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:56.123018 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:56.123018 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:56.123018 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:56.123018 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:56.123018 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:56.123018 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:56.123018 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:56.123018 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:56.123018 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:56.123018 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:56.123018 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:56.124089 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:56.124089 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:56.124089 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:56.124089 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:56.124089 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:56.124089 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:56.124089 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:56.124089 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:56.124089 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:56.124089 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:56.124089 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:56.124089 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:56.124089 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:56.124089 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:56.124089 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:56.124089 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:56.124089 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:56.125212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:56.125212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:56.125212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:56.125212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:56.127749 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 487/967] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:56.133248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c1161a-ac83-4b55-864d-af6be69ae4e5 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:56.243477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2c5655-4b7c-4521-a4d9-844ded9563fa None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:56.246488 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:56.263055 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c1161a-ac83-4b55-864d-af6be69ae4e5 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:56.286721 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff19632-bba7-4a50-8e83-b3792fa2e918 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:56.287651 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff19632-bba7-4a50-8e83-b3792fa2e918 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:56.287996 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff19632-bba7-4a50-8e83-b3792fa2e918 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:56.288340 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff19632-bba7-4a50-8e83-b3792fa2e918 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:56.289060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff19632-bba7-4a50-8e83-b3792fa2e918 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:56.291088 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff19632-bba7-4a50-8e83-b3792fa2e918 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:56.305618 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e74e9-d3df-435a-9eee-2ad741611450 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:56.306814 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e74e9-d3df-435a-9eee-2ad741611450 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:56.307303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e74e9-d3df-435a-9eee-2ad741611450 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:56.307582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-6ff19632-bba7-4a50-8e83-b3792fa2e918 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 16:54:56.307582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:56.307582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:54:56.307582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:54:56.307582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:54:56.307582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:54:56.307582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:54:56.307582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:56.307582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 16:54:56.307582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:56.307582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:54:56.307582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:56.307582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:56.307582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:56.307582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:56.307582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:56.307582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:56.308585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:56.308585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:56.308585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:56.308585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:56.308585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:56.308585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:56.308585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:54:56.308585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:54:56.308585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:54:56.308585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:54:56.308585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:56.308585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:56.308585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:56.308585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:56.308585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:56.308585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:56.309427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:56.309427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:54:56.309427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:56.309427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:56.309427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:56.309427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:56.309427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:56.309427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:56.309427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:56.309427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:56.309427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:54:56.309427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:56.309427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 16:54:56.309427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:56.310995 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-715e74e9-d3df-435a-9eee-2ad741611450 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:56.311944 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715e74e9-d3df-435a-9eee-2ad741611450 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:56.312459 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715e74e9-d3df-435a-9eee-2ad741611450 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:56.313891 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 481/968] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 23 16:54:56 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:54:56.314365 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-715e74e9-d3df-435a-9eee-2ad741611450 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:56.314822 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff19632-bba7-4a50-8e83-b3792fa2e918 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:56.330578 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 488/969] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:56.336112 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e74e9-d3df-435a-9eee-2ad741611450 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:56.371451 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:56.442013 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff19632-bba7-4a50-8e83-b3792fa2e918 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:56.460972 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-715e74e9-d3df-435a-9eee-2ad741611450 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:56.487551 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03979313-e08e-4939-9087-7f173fbad1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:56.488227 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03979313-e08e-4939-9087-7f173fbad1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:56.488506 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03979313-e08e-4939-9087-7f173fbad1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:56.488833 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03979313-e08e-4939-9087-7f173fbad1a5 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:56.489732 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03979313-e08e-4939-9087-7f173fbad1a5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:56.490407 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03979313-e08e-4939-9087-7f173fbad1a5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:56.491815 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03979313-e08e-4939-9087-7f173fbad1a5 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:56.499054 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 482/970] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 23 16:54:56 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:56.505180 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03979313-e08e-4939-9087-7f173fbad1a5 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:56.514506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a59c8-b7fe-4db3-a0ba-7910f8eeb15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:56.515374 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a59c8-b7fe-4db3-a0ba-7910f8eeb15d None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:56.515655 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a59c8-b7fe-4db3-a0ba-7910f8eeb15d None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:56.515929 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8a59c8-b7fe-4db3-a0ba-7910f8eeb15d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:56.523772 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a59c8-b7fe-4db3-a0ba-7910f8eeb15d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:56.525222 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8a59c8-b7fe-4db3-a0ba-7910f8eeb15d None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:56.540523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-1e8a59c8-b7fe-4db3-a0ba-7910f8eeb15d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 16:54:56.540523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:56.540523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:56.540523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:56.540523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:56.540523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:56.540523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:56.540523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:56.540523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:56.540523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:56.540523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:56.540523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:56.540523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:56.540523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:56.540523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:56.540523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:56.541499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:56.541499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:56.541499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:56.541499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:56.541499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:56.541499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:56.541499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:56.541499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:56.541499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:56.541499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:56.541499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:56.541499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:56.541499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:56.541499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:56.541499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:56.541499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:56.541499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:56.541499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:56.542419 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:56.542419 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:56.542419 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:56.542419 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:56.542419 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:56.542419 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:56.542419 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:56.542419 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:56.542419 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 16:54:56.542419 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:56.546628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 489/971] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 23 16:54:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:54:56.547623 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a59c8-b7fe-4db3-a0ba-7910f8eeb15d None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:56.559020 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:56.631659 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03979313-e08e-4939-9087-7f173fbad1a5 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:56.642276 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:56.663654 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8a59c8-b7fe-4db3-a0ba-7910f8eeb15d None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:56.670454 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feba579-6f7f-4e16-9ff3-e59315f3b9f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:56.671426 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feba579-6f7f-4e16-9ff3-e59315f3b9f9 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:56.671902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feba579-6f7f-4e16-9ff3-e59315f3b9f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:56.672364 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5feba579-6f7f-4e16-9ff3-e59315f3b9f9 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:56.673252 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feba579-6f7f-4e16-9ff3-e59315f3b9f9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:56.673765 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feba579-6f7f-4e16-9ff3-e59315f3b9f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:56.675437 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5feba579-6f7f-4e16-9ff3-e59315f3b9f9 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:56.685739 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 483/972] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 23 16:54:56 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:56.686869 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feba579-6f7f-4e16-9ff3-e59315f3b9f9 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:56.700203 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe9a05-1116-4bfd-9054-bbb8a30f2b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:56.700953 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe9a05-1116-4bfd-9054-bbb8a30f2b6e None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:56.701158 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe9a05-1116-4bfd-9054-bbb8a30f2b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:56.701434 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afe9a05-1116-4bfd-9054-bbb8a30f2b6e None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:56.702430 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe9a05-1116-4bfd-9054-bbb8a30f2b6e None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:56.702788 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe9a05-1116-4bfd-9054-bbb8a30f2b6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:56.704226 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afe9a05-1116-4bfd-9054-bbb8a30f2b6e None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:56.723310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 490/973] 10.222.0.13 () {56 vars in 1149 bytes} [Mon May 23 16:54:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:54:56.728818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe9a05-1116-4bfd-9054-bbb8a30f2b6e None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:56.795052 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5feba579-6f7f-4e16-9ff3-e59315f3b9f9 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:56.798063 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:56.832257 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a6bbd5-1a7b-4fa6-a836-23e7ccb42d9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:56.832917 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6bbd5-1a7b-4fa6-a836-23e7ccb42d9f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:56.833197 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6bbd5-1a7b-4fa6-a836-23e7ccb42d9f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:56.833470 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a6bbd5-1a7b-4fa6-a836-23e7ccb42d9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:56.834260 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6bbd5-1a7b-4fa6-a836-23e7ccb42d9f None admin] RBAC: Adding query filter params (interface=admin, service_id=fe5a4b5bcc57490ab5f97e67634ba850, region_id=RegionOne) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:56.834578 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6bbd5-1a7b-4fa6-a836-23e7ccb42d9f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:56.835987 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a6bbd5-1a7b-4fa6-a836-23e7ccb42d9f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:56.836904 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afe9a05-1116-4bfd-9054-bbb8a30f2b6e None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:56.841060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 484/974] 10.222.0.13 () {56 vars in 1249 bytes} [Mon May 23 16:54:56 2022] GET /identity/v3/endpoints?service_id=fe5a4b5bcc57490ab5f97e67634ba850&interface=admin®ion_id=RegionOne => generated 193 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:56.842268 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f3617-652e-4fb3-a9ca-6157d3854ff6 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:56.842548 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f3617-652e-4fb3-a9ca-6157d3854ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:56.842817 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891f3617-652e-4fb3-a9ca-6157d3854ff6 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:56.847077 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 485/975] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:56.848634 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f3617-652e-4fb3-a9ca-6157d3854ff6 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:56.874523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2937379-8d60-4736-8c33-593ed2beac9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:56.875212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2937379-8d60-4736-8c33-593ed2beac9b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:56.875494 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2937379-8d60-4736-8c33-593ed2beac9b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:56.875768 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2937379-8d60-4736-8c33-593ed2beac9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:56.888274 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:56.953313 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2937379-8d60-4736-8c33-593ed2beac9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:56.955479 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2937379-8d60-4736-8c33-593ed2beac9b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:56.958055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891f3617-652e-4fb3-a9ca-6157d3854ff6 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:56.993561 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c57b5-3eec-446d-8baa-8c512172e36d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:56.994224 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c57b5-3eec-446d-8baa-8c512172e36d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:56.994499 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c57b5-3eec-446d-8baa-8c512172e36d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:56.994772 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c57b5-3eec-446d-8baa-8c512172e36d None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:56.995470 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c57b5-3eec-446d-8baa-8c512172e36d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:56.996845 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c57b5-3eec-446d-8baa-8c512172e36d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:57.003174 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 486/976] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 23 16:54:56 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:54:57.004182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbecda2-d1e4-4e27-9194-4b6bd84f1214 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:57.004464 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbecda2-d1e4-4e27-9194-4b6bd84f1214 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:57.004733 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbecda2-d1e4-4e27-9194-4b6bd84f1214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:57.022897 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-8fbecda2-d1e4-4e27-9194-4b6bd84f1214 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:57.024171 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8fbecda2-d1e4-4e27-9194-4b6bd84f1214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:57.034542 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:57.090687 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 491/977] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:54:56 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 16:54:57.096960 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2937379-8d60-4736-8c33-593ed2beac9b None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:57.111442 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 487/978] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:54:57.130487 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbecda2-d1e4-4e27-9194-4b6bd84f1214 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:57.163679 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:57.210067 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2937379-8d60-4736-8c33-593ed2beac9b None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:57.244704 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbecda2-d1e4-4e27-9194-4b6bd84f1214 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:57.245798 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f690cf2-c325-4c1f-8aab-f238fe8618cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:57.246607 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f690cf2-c325-4c1f-8aab-f238fe8618cd None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:57.246942 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f690cf2-c325-4c1f-8aab-f238fe8618cd None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:57.247811 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f690cf2-c325-4c1f-8aab-f238fe8618cd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:57.259218 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f690cf2-c325-4c1f-8aab-f238fe8618cd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:57.261833 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f690cf2-c325-4c1f-8aab-f238fe8618cd None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:57.280426 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d57bf3-0bfd-4544-b9f6-1429984a5f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:57.281097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d57bf3-0bfd-4544-b9f6-1429984a5f0a None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:57.281371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d57bf3-0bfd-4544-b9f6-1429984a5f0a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:57.281641 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d57bf3-0bfd-4544-b9f6-1429984a5f0a None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:57.282325 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d57bf3-0bfd-4544-b9f6-1429984a5f0a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:57.283839 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d57bf3-0bfd-4544-b9f6-1429984a5f0a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:57.284773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-5f690cf2-c325-4c1f-8aab-f238fe8618cd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:57.284773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:57.284773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:57.284773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:57.284773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:57.284773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:57.284773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:57.284773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:57.284773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:57.284773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:57.284773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:57.284773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:57.284773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:54:57.284773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:54:57.284773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:54:57.285660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:54:57.285660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:57.285660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:57.285660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:57.285660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:57.285660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:57.285660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:57.285660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:57.285660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:57.285660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:57.285660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:57.285660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:57.285660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:57.285660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:57.285660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:57.285660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:57.285660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:57.286817 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:54:57.286817 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:54:57.286817 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:54:57.286817 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:57.286817 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 492/979] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:54:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:57.287344 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87091995-3742-4727-a2e0-55494b392a9b None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:57.287526 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87091995-3742-4727-a2e0-55494b392a9b None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:57.287764 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87091995-3742-4727-a2e0-55494b392a9b None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:57.289164 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 493/980] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:57 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:57.294207 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01ae82-e229-4547-9043-bfc0c5895265 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:57.294489 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01ae82-e229-4547-9043-bfc0c5895265 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:57.294764 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de01ae82-e229-4547-9043-bfc0c5895265 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:57.303005 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.sql.core [None req-31d57bf3-0bfd-4544-b9f6-1429984a5f0a None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 23 16:54:57.303005 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 23 16:54:57.303005 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [parameters: {'user_id': '4ce8715eebd04bca8f8909b35741b04b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 23 16:54:57.303005 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80464) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 23 16:54:57.310790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-31d57bf3-0bfd-4544-b9f6-1429984a5f0a None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 23 16:54:57.310790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:57.310790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 16:54:57.310790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 16:54:57.310790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 16:54:57.310790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 16:54:57.310790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 16:54:57.310790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application result = self._query(query) May 23 16:54:57.310790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 16:54:57.310790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application conn.query(q) May 23 16:54:57.310790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 16:54:57.310790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 16:54:57.310790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 16:54:57.310790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application result.read() May 23 16:54:57.310790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 16:54:57.311812 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 16:54:57.311812 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 16:54:57.311812 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 16:54:57.311812 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 16:54:57.311812 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 16:54:57.311812 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 16:54:57.311812 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 16:54:57.311812 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 23 16:54:57.311812 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:57.311812 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 23 16:54:57.311812 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:57.311812 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:57.311812 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 23 16:54:57.311812 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 23 16:54:57.311812 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 23 16:54:57.311812 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 23 16:54:57.311812 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 16:54:57.311812 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application next(self.gen) May 23 16:54:57.312779 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 23 16:54:57.312779 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application yield resource May 23 16:54:57.312779 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 16:54:57.312779 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application next(self.gen) May 23 16:54:57.312779 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 23 16:54:57.312779 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self.session.rollback() May 23 16:54:57.312779 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 23 16:54:57.312779 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self.force_reraise() May 23 16:54:57.312779 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 23 16:54:57.312779 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise self.value May 23 16:54:57.312779 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 23 16:54:57.312779 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 23 16:54:57.312779 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 23 16:54:57.312779 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application session.commit() May 23 16:54:57.312779 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 23 16:54:57.312779 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 23 16:54:57.312779 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 23 16:54:57.317020 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._prepare_impl() May 23 16:54:57.317020 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 23 16:54:57.317020 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self.session.flush() May 23 16:54:57.317020 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 23 16:54:57.317020 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._flush(objects) May 23 16:54:57.317020 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 23 16:54:57.317020 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 23 16:54:57.317020 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 23 16:54:57.317020 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application compat.raise_( May 23 16:54:57.317020 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 16:54:57.317020 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception May 23 16:54:57.317020 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 23 16:54:57.317020 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application flush_context.execute() May 23 16:54:57.317020 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 23 16:54:57.317020 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rec.execute(self) May 23 16:54:57.317020 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 23 16:54:57.317020 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 23 16:54:57.319289 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 23 16:54:57.319289 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application _emit_insert_statements( May 23 16:54:57.319289 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 23 16:54:57.319289 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application result = connection._execute_20( May 23 16:54:57.319289 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 23 16:54:57.319289 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 23 16:54:57.319289 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 23 16:54:57.319289 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 23 16:54:57.319289 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 23 16:54:57.319289 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ret = self._execute_context( May 23 16:54:57.319289 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 23 16:54:57.319289 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 23 16:54:57.319289 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 23 16:54:57.319289 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 23 16:54:57.319289 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 16:54:57.319289 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception May 23 16:54:57.321875 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 16:54:57.321875 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 16:54:57.321875 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 16:54:57.321875 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 16:54:57.321875 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 16:54:57.321875 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application result = self._query(query) May 23 16:54:57.321875 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 16:54:57.321875 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application conn.query(q) May 23 16:54:57.321875 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 16:54:57.321875 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 16:54:57.321875 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 16:54:57.321875 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application result.read() May 23 16:54:57.321875 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 16:54:57.321875 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 16:54:57.321875 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 16:54:57.321875 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 16:54:57.325371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 16:54:57.325371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 16:54:57.325371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 16:54:57.325371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 16:54:57.325371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 23 16:54:57.325371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 23 16:54:57.325371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [parameters: {'user_id': '4ce8715eebd04bca8f8909b35741b04b', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 23 16:54:57.325371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 23 16:54:57.325371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:57.325371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:54:57.325371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:57.325371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:57.325371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:57.325371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:57.325371 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:57.327649 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:57.327649 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:57.327649 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:57.327649 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:57.327649 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:57.327649 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:57.327649 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:57.327649 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 23 16:54:57.327649 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 23 16:54:57.327649 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:57.327649 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:57.327649 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:57.327649 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:57.327649 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:57.327649 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:57.327649 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 23 16:54:57.327649 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 23 16:54:57.330356 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 23 16:54:57.330356 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 23 16:54:57.330356 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 23 16:54:57.330356 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 23 16:54:57.330356 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 23 16:54:57.330356 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:54:57.331394 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-de01ae82-e229-4547-9043-bfc0c5895265 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:57.331394 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-de01ae82-e229-4547-9043-bfc0c5895265 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:57.332085 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 488/981] 10.222.0.13 () {60 vars in 1131 bytes} [Mon May 23 16:54:57 2022] POST /identity/v3/users => generated 160 bytes in 182 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 23 16:54:57.332085 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d57bf3-0bfd-4544-b9f6-1429984a5f0a None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:57.332353 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:57.383277 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 494/982] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:57 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 16:54:57.384444 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01ae82-e229-4547-9043-bfc0c5895265 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:57.421997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:57.432917 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d57bf3-0bfd-4544-b9f6-1429984a5f0a None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:57.471902 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcfd23b-ac1d-49d6-955d-9e8eec55dedf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:57.472749 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfd23b-ac1d-49d6-955d-9e8eec55dedf None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:57.473505 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfd23b-ac1d-49d6-955d-9e8eec55dedf None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:57.473891 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcfd23b-ac1d-49d6-955d-9e8eec55dedf None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:57.474712 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcfd23b-ac1d-49d6-955d-9e8eec55dedf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:57.475146 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcfd23b-ac1d-49d6-955d-9e8eec55dedf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:57.476884 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcfd23b-ac1d-49d6-955d-9e8eec55dedf None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:57.493143 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 489/983] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:54:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:57.493993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcfd23b-ac1d-49d6-955d-9e8eec55dedf None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:57.511453 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de01ae82-e229-4547-9043-bfc0c5895265 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:57.548510 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783f23e-38a0-4e87-9032-1b14c8ee0872 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:57.549678 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783f23e-38a0-4e87-9032-1b14c8ee0872 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:57.550360 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783f23e-38a0-4e87-9032-1b14c8ee0872 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:57.550641 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e783f23e-38a0-4e87-9032-1b14c8ee0872 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:57.558359 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783f23e-38a0-4e87-9032-1b14c8ee0872 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:57.559833 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e783f23e-38a0-4e87-9032-1b14c8ee0872 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:57.574524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-e783f23e-38a0-4e87-9032-1b14c8ee0872 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 16:54:57.574524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:57.574524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:57.574524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:57.574524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:57.574524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:57.574524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:57.574524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:57.574524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:57.574524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:57.574524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:57.574524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:57.574524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:54:57.574524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:54:57.574524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:54:57.574524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:54:57.575692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:57.575692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:57.575692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:54:57.575692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:57.575692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:54:57.575692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:54:57.575692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:57.575692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:57.575692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:57.575692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:57.575692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:57.575692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:57.575692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:57.575692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:57.575692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:57.575692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:57.575692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:57.575692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:57.576936 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:57.576936 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:57.576936 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:54:57.576936 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:54:57.576936 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:54:57.576936 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:54:57.576936 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:54:57.576936 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:54:57.576936 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 16:54:57.576936 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:57.580895 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 495/984] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:54:57 2022] GET /identity/v3/users/cinder => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:57.581757 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783f23e-38a0-4e87-9032-1b14c8ee0872 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:57.594783 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:57.603278 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcfd23b-ac1d-49d6-955d-9e8eec55dedf None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:57.642006 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e834a-9522-42d0-84ec-25187212e6a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:57.642696 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e834a-9522-42d0-84ec-25187212e6a2 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:57.643075 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e834a-9522-42d0-84ec-25187212e6a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:57.643407 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d00e834a-9522-42d0-84ec-25187212e6a2 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:57.644184 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00e834a-9522-42d0-84ec-25187212e6a2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:57.645620 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d00e834a-9522-42d0-84ec-25187212e6a2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:57.661354 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 490/985] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 23 16:54:57 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:54:57.662331 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e834a-9522-42d0-84ec-25187212e6a2 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:57.691164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:57.699872 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e783f23e-38a0-4e87-9032-1b14c8ee0872 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:57.751025 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8259e304-4cb9-46be-ba80-43d548034ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:57.752166 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259e304-4cb9-46be-ba80-43d548034ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:57.752593 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259e304-4cb9-46be-ba80-43d548034ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:57.753577 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8259e304-4cb9-46be-ba80-43d548034ef6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:57.765187 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8259e304-4cb9-46be-ba80-43d548034ef6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:57.766832 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8259e304-4cb9-46be-ba80-43d548034ef6 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:57.787643 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-8259e304-4cb9-46be-ba80-43d548034ef6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:57.787643 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:57.787643 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:57.787643 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:57.787643 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:57.787643 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:57.787643 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:57.787643 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:57.787643 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:57.787643 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:57.787643 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:57.787643 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:57.787643 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:54:57.787643 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:54:57.787643 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:54:57.788740 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:54:57.788740 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:57.788740 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:57.788740 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:57.788740 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:57.788740 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:57.788740 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:57.788740 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:57.788740 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:57.788740 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:57.788740 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:57.788740 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:57.788740 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:57.788740 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:57.788740 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:57.788740 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:57.788740 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:57.789813 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:54:57.789813 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:54:57.789813 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:54:57.789813 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:54:57.789813 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:54:57.789813 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:54:57.789813 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:54:57.789813 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:57.794628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 496/986] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:54:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:57.795611 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8259e304-4cb9-46be-ba80-43d548034ef6 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:57.815847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d00e834a-9522-42d0-84ec-25187212e6a2 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:57.864105 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc97ff-5e19-4628-b84a-2b42fbe00174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:57.864907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc97ff-5e19-4628-b84a-2b42fbe00174 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:57.865324 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc97ff-5e19-4628-b84a-2b42fbe00174 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:57.865605 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc97ff-5e19-4628-b84a-2b42fbe00174 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:57.866411 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc97ff-5e19-4628-b84a-2b42fbe00174 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:57.866979 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc97ff-5e19-4628-b84a-2b42fbe00174 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:57.868821 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc97ff-5e19-4628-b84a-2b42fbe00174 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:57.872606 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:57.912980 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 491/987] 10.222.0.13 () {56 vars in 1101 bytes} [Mon May 23 16:54:57 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:57.923686 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8259e304-4cb9-46be-ba80-43d548034ef6 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:57.968164 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c7eea2-d839-4d2e-ac4b-7e4e26cae5c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:57.968819 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7eea2-d839-4d2e-ac4b-7e4e26cae5c3 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:57.969182 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7eea2-d839-4d2e-ac4b-7e4e26cae5c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:57.969464 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c7eea2-d839-4d2e-ac4b-7e4e26cae5c3 None admin] PATH_INFO: `/v3/projects` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:57.970263 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7eea2-d839-4d2e-ac4b-7e4e26cae5c3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:57.970576 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7eea2-d839-4d2e-ac4b-7e4e26cae5c3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:57.972151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c7eea2-d839-4d2e-ac4b-7e4e26cae5c3 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:57.986938 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 497/988] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:54:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:57.996090 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc97ff-5e19-4628-b84a-2b42fbe00174 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:58.073304 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:58.106788 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc97ff-5e19-4628-b84a-2b42fbe00174 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:58.144574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9f40e1-0afc-46ea-9827-22a7d5afa02e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:58.145233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f40e1-0afc-46ea-9827-22a7d5afa02e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:58.145511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f40e1-0afc-46ea-9827-22a7d5afa02e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:58.145781 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9f40e1-0afc-46ea-9827-22a7d5afa02e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:58.146602 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9f40e1-0afc-46ea-9827-22a7d5afa02e None admin] RBAC: Adding query filter params (role.id=7141f022bbca499c80cc11cca858e064, scope.project.id=0595667fb2e04a3ca4716f669d6698b0, user.id=4b2d638e6bd240ccbe45228352900bdb) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:58.146934 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9f40e1-0afc-46ea-9827-22a7d5afa02e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:58.148367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9f40e1-0afc-46ea-9827-22a7d5afa02e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:58.154762 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 492/989] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:54:57 2022] GET /identity/v3/role_assignments?user.id=4b2d638e6bd240ccbe45228352900bdb&scope.project.id=0595667fb2e04a3ca4716f669d6698b0&role.id=7141f022bbca499c80cc11cca858e064 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:59.261769 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177579d-d57a-4234-8e2a-4ea3dfabf82a None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:59.262824 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177579d-d57a-4234-8e2a-4ea3dfabf82a None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:59.263507 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c177579d-d57a-4234-8e2a-4ea3dfabf82a None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:59.265347 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 498/990] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:59.270562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d94e6-8aea-487f-98bf-82449658b892 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:59.271447 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d94e6-8aea-487f-98bf-82449658b892 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:59.271927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736d94e6-8aea-487f-98bf-82449658b892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:59.291212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-736d94e6-8aea-487f-98bf-82449658b892 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:59.292797 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-736d94e6-8aea-487f-98bf-82449658b892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:54:59.362979 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 493/991] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:59 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:54:59.385199 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177579d-d57a-4234-8e2a-4ea3dfabf82a None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:59.497067 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c177579d-d57a-4234-8e2a-4ea3dfabf82a None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:59.521462 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:59.533436 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c721e-8bd2-435d-8181-1d3910b8a352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:59.534259 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c721e-8bd2-435d-8181-1d3910b8a352 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:59.534537 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c721e-8bd2-435d-8181-1d3910b8a352 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:59.534810 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b2c721e-8bd2-435d-8181-1d3910b8a352 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:59.535499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c721e-8bd2-435d-8181-1d3910b8a352 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:59.536868 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b2c721e-8bd2-435d-8181-1d3910b8a352 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:59.548907 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-4b2c721e-8bd2-435d-8181-1d3910b8a352 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 16:54:59.548907 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:59.548907 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:54:59.548907 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:54:59.548907 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:54:59.548907 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:54:59.548907 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:54:59.548907 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:59.548907 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 16:54:59.548907 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:59.548907 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:54:59.548907 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:59.548907 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:54:59.548907 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:54:59.548907 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:54:59.548907 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:54:59.548907 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:54:59.550093 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:54:59.550093 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:54:59.550093 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:54:59.550093 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:54:59.550093 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:54:59.550093 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:54:59.550093 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:54:59.550093 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:54:59.550093 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:54:59.550093 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:54:59.550093 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:54:59.550093 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:54:59.550093 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:54:59.550093 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:54:59.550093 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:54:59.550093 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:54:59.551413 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:54:59.551413 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:54:59.551413 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:54:59.551413 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:54:59.551413 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:54:59.551413 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:54:59.551413 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:54:59.551413 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:54:59.551413 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:54:59.551413 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:54:59.551413 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:54:59.551413 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:54:59.551413 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 16:54:59.551413 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:54:59.555347 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 499/992] 10.222.0.13 () {56 vars in 1126 bytes} [Mon May 23 16:54:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:54:59.556517 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d94e6-8aea-487f-98bf-82449658b892 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:59.622452 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:54:59.663457 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736d94e6-8aea-487f-98bf-82449658b892 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:59.697966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c234f5-b889-42db-9ddb-973019704a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:59.698630 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c234f5-b889-42db-9ddb-973019704a41 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:59.698922 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c234f5-b889-42db-9ddb-973019704a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:59.699217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c234f5-b889-42db-9ddb-973019704a41 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:59.699911 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c234f5-b889-42db-9ddb-973019704a41 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:59.700217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c234f5-b889-42db-9ddb-973019704a41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:59.701593 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c234f5-b889-42db-9ddb-973019704a41 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:59.706700 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 494/993] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 23 16:54:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:59.712427 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c721e-8bd2-435d-8181-1d3910b8a352 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:59.823104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b2c721e-8bd2-435d-8181-1d3910b8a352 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:59.857026 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9d6dc-5798-4cc7-8550-411036a2dd6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:54:59.857696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9d6dc-5798-4cc7-8550-411036a2dd6d None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:59.857976 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9d6dc-5798-4cc7-8550-411036a2dd6d None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:59.858247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9d6dc-5798-4cc7-8550-411036a2dd6d None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:59.858980 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9d6dc-5798-4cc7-8550-411036a2dd6d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:54:59.859285 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9d6dc-5798-4cc7-8550-411036a2dd6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:54:59.860658 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9d6dc-5798-4cc7-8550-411036a2dd6d None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:54:59.865938 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 500/994] 10.222.0.13 () {56 vars in 1122 bytes} [Mon May 23 16:54:59 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:54:59.871463 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c234f5-b889-42db-9ddb-973019704a41 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:54:59.956213 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b10f3a-c125-45a3-9959-3e5de33c9669 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:59.956554 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b10f3a-c125-45a3-9959-3e5de33c9669 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:59.956887 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b10f3a-c125-45a3-9959-3e5de33c9669 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:59.958512 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 501/995] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:54:59.963566 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109fe6e-0d86-4f29-9e5d-d85587e37627 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:54:59.963973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109fe6e-0d86-4f29-9e5d-d85587e37627 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:54:59.964257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b109fe6e-0d86-4f29-9e5d-d85587e37627 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:54:59.976527 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c234f5-b889-42db-9ddb-973019704a41 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:54:59.982229 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-b109fe6e-0d86-4f29-9e5d-d85587e37627 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:54:59.983455 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-b109fe6e-0d86-4f29-9e5d-d85587e37627 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:00.005064 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:00.014488 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4fb741-2a48-4eac-82d9-bff404a23292 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:00.015427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fb741-2a48-4eac-82d9-bff404a23292 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:00.015722 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fb741-2a48-4eac-82d9-bff404a23292 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:00.016061 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4fb741-2a48-4eac-82d9-bff404a23292 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:00.016920 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4fb741-2a48-4eac-82d9-bff404a23292 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:00.018597 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4fb741-2a48-4eac-82d9-bff404a23292 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:00.033577 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:00.037004 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 495/996] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 23 16:54:59 2022] POST /identity/v3/endpoints => generated 325 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:55:00.071370 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 502/997] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:54:59 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:55:00.094958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4fb741-2a48-4eac-82d9-bff404a23292 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:00.210090 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4fb741-2a48-4eac-82d9-bff404a23292 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:00.253401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece723ac-b27b-4680-ac3e-cd721cefc08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:00.254210 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece723ac-b27b-4680-ac3e-cd721cefc08b None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:00.254483 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece723ac-b27b-4680-ac3e-cd721cefc08b None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:00.254747 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece723ac-b27b-4680-ac3e-cd721cefc08b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:00.255459 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece723ac-b27b-4680-ac3e-cd721cefc08b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:00.256848 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece723ac-b27b-4680-ac3e-cd721cefc08b None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:00.269044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-ece723ac-b27b-4680-ac3e-cd721cefc08b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 23 16:55:00.269044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:00.269044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:55:00.269044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:55:00.269044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:55:00.269044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:55:00.269044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:55:00.269044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:55:00.269044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 16:55:00.269044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:00.269044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:55:00.269044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:00.269044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:00.269044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:00.269044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:00.269044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:00.269044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:00.270056 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:00.270056 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:00.270056 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:00.270056 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:00.270056 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:00.270056 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:00.270056 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:55:00.270056 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:55:00.270056 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:55:00.270056 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:55:00.270056 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:00.270056 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:00.270056 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:00.270056 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:00.270056 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:00.270056 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:00.271477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:00.271477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:55:00.271477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:00.271477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:00.271477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:00.271477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:00.271477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:00.271477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:00.271477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:00.271477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:00.271477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:55:00.271477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:55:00.271477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 16:55:00.271477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:00.271477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 496/998] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:55:00 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:55:00.277317 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109fe6e-0d86-4f29-9e5d-d85587e37627 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:00.384472 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b109fe6e-0d86-4f29-9e5d-d85587e37627 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:00.400742 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db97166-39d0-408a-be56-c7db0e30c208 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:00.401037 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db97166-39d0-408a-be56-c7db0e30c208 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:00.401332 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db97166-39d0-408a-be56-c7db0e30c208 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:00.402893 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 497/999] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:55:00.408066 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b5ced-9b8d-4ea9-a773-3f7595b6d0ce None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:00.408359 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b5ced-9b8d-4ea9-a773-3f7595b6d0ce None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:00.408635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5b5ced-9b8d-4ea9-a773-3f7595b6d0ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:00.423130 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793dfe4-39f1-4a42-adfc-0935db0e1033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:00.424082 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793dfe4-39f1-4a42-adfc-0935db0e1033 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:00.424298 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793dfe4-39f1-4a42-adfc-0935db0e1033 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:00.424575 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8793dfe4-39f1-4a42-adfc-0935db0e1033 None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:00.425405 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8793dfe4-39f1-4a42-adfc-0935db0e1033 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:00.425730 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8793dfe4-39f1-4a42-adfc-0935db0e1033 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:00.427375 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8793dfe4-39f1-4a42-adfc-0935db0e1033 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:00.431752 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-bf5b5ced-9b8d-4ea9-a773-3f7595b6d0ce None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:00.432620 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 503/1000] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 23 16:55:00 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:55:00.433276 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-bf5b5ced-9b8d-4ea9-a773-3f7595b6d0ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:00.439975 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793dfe4-39f1-4a42-adfc-0935db0e1033 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:00.487240 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:00.492905 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:00.516835 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 498/1001] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:00 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:55:00.538598 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b5ced-9b8d-4ea9-a773-3f7595b6d0ce None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:00.550785 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8793dfe4-39f1-4a42-adfc-0935db0e1033 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:00.596902 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd1928-adaf-4fa6-b0ac-fed82242f229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:00.597611 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd1928-adaf-4fa6-b0ac-fed82242f229 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:00.597892 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd1928-adaf-4fa6-b0ac-fed82242f229 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:00.598161 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fd1928-adaf-4fa6-b0ac-fed82242f229 None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:00.598858 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd1928-adaf-4fa6-b0ac-fed82242f229 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:00.599249 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd1928-adaf-4fa6-b0ac-fed82242f229 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:00.601367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fd1928-adaf-4fa6-b0ac-fed82242f229 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:00.612943 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 504/1002] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 23 16:55:00 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:00.614026 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd1928-adaf-4fa6-b0ac-fed82242f229 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:00.655553 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5b5ced-9b8d-4ea9-a773-3f7595b6d0ce None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:00.691480 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848eed89-bef7-49a5-872a-2b9fb586f663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:00.692276 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848eed89-bef7-49a5-872a-2b9fb586f663 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:00.692560 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848eed89-bef7-49a5-872a-2b9fb586f663 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:00.692832 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848eed89-bef7-49a5-872a-2b9fb586f663 None admin] PATH_INFO: `/v3/services/image` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:00.693481 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848eed89-bef7-49a5-872a-2b9fb586f663 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:00.694870 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848eed89-bef7-49a5-872a-2b9fb586f663 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:00.706787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-848eed89-bef7-49a5-872a-2b9fb586f663 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 23 16:55:00.706787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:00.706787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:55:00.706787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:55:00.706787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:55:00.706787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:55:00.706787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:55:00.706787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:55:00.706787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 16:55:00.706787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:00.706787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:55:00.706787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:00.706787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:00.706787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:00.706787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:00.706787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:00.706787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:00.707857 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:00.707857 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:00.707857 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:00.707857 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:00.707857 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:00.707857 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:00.707857 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:55:00.707857 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:55:00.707857 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:55:00.707857 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:55:00.707857 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:00.707857 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:00.707857 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:00.707857 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:00.707857 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:00.707857 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:00.708692 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:00.708692 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:55:00.708692 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:00.708692 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:00.708692 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:00.708692 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:00.708692 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:00.708692 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:00.708692 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:00.708692 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:00.708692 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:55:00.708692 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:55:00.708692 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 16:55:00.708692 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:00.708692 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 499/1003] 10.222.0.13 () {56 vars in 1105 bytes} [Mon May 23 16:55:00 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:55:00.714089 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848eed89-bef7-49a5-872a-2b9fb586f663 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:00.721303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fd1928-adaf-4fa6-b0ac-fed82242f229 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:00.729187 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:00.762660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3cdc1-7ae7-4214-a24d-c6ba9f044f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:00.763357 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3cdc1-7ae7-4214-a24d-c6ba9f044f80 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:00.763642 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3cdc1-7ae7-4214-a24d-c6ba9f044f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:00.764194 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db3cdc1-7ae7-4214-a24d-c6ba9f044f80 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:00.764682 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:00.764793 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3cdc1-7ae7-4214-a24d-c6ba9f044f80 None admin] RBAC: Adding query filter params (interface=public, service_id=37f7a00498734f11ac1862637564afd0, region_id=RegionOne) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:00.765060 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3cdc1-7ae7-4214-a24d-c6ba9f044f80 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:00.766389 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db3cdc1-7ae7-4214-a24d-c6ba9f044f80 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:00.772320 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 505/1004] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 23 16:55:00 2022] GET /identity/v3/endpoints?service_id=37f7a00498734f11ac1862637564afd0&interface=public®ion_id=RegionOne => generated 194 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:00.777865 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3cdc1-7ae7-4214-a24d-c6ba9f044f80 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:00.825872 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848eed89-bef7-49a5-872a-2b9fb586f663 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:00.864827 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726818b6-27b2-4b8c-acf4-0dc237dcb1df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:00.865730 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726818b6-27b2-4b8c-acf4-0dc237dcb1df None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:00.866088 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726818b6-27b2-4b8c-acf4-0dc237dcb1df None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:00.866422 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726818b6-27b2-4b8c-acf4-0dc237dcb1df None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:00.867219 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726818b6-27b2-4b8c-acf4-0dc237dcb1df None admin] RBAC: Adding query filter params (name=image) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:00.867584 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726818b6-27b2-4b8c-acf4-0dc237dcb1df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:00.869119 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726818b6-27b2-4b8c-acf4-0dc237dcb1df None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:00.874220 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 500/1005] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:55:00 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:55:00.880570 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726818b6-27b2-4b8c-acf4-0dc237dcb1df None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:00.893251 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db3cdc1-7ae7-4214-a24d-c6ba9f044f80 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:00.936127 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ea708f-cd75-4dbd-b7c5-7ad343fa5dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:00.936715 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea708f-cd75-4dbd-b7c5-7ad343fa5dea None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:00.937022 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea708f-cd75-4dbd-b7c5-7ad343fa5dea None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:00.937267 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ea708f-cd75-4dbd-b7c5-7ad343fa5dea None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:00.937951 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea708f-cd75-4dbd-b7c5-7ad343fa5dea None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:00.939860 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35ea708f-cd75-4dbd-b7c5-7ad343fa5dea None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:00.954941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 506/1006] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 23 16:55:00 2022] GET /identity/v3/services => generated 1727 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 16:55:01.014547 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726818b6-27b2-4b8c-acf4-0dc237dcb1df None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:01.026658 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:01.056118 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344591c5-f2e4-43e5-9ab0-adf4ebf42068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:01.056893 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344591c5-f2e4-43e5-9ab0-adf4ebf42068 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:01.057241 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344591c5-f2e4-43e5-9ab0-adf4ebf42068 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:01.057579 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344591c5-f2e4-43e5-9ab0-adf4ebf42068 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:01.058365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344591c5-f2e4-43e5-9ab0-adf4ebf42068 None admin] RBAC: Adding query filter params (type=image) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:01.058751 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344591c5-f2e4-43e5-9ab0-adf4ebf42068 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:01.060270 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344591c5-f2e4-43e5-9ab0-adf4ebf42068 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:01.067335 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 501/1007] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:55:00 2022] GET /identity/v3/services?type=image => generated 122 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:02.255511 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74aaf94-5c1f-4462-9a49-e62d00f07ee2 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:02.256051 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74aaf94-5c1f-4462-9a49-e62d00f07ee2 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:02.256051 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74aaf94-5c1f-4462-9a49-e62d00f07ee2 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:02.257964 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 507/1008] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:55:02.263448 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549476e-287a-401a-b44e-7885e56481ee None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:02.263877 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549476e-287a-401a-b44e-7885e56481ee None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:02.264067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5549476e-287a-401a-b44e-7885e56481ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:02.281818 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-5549476e-287a-401a-b44e-7885e56481ee None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:02.283336 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5549476e-287a-401a-b44e-7885e56481ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:02.355695 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 502/1009] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:02 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:55:02.378684 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74aaf94-5c1f-4462-9a49-e62d00f07ee2 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:02.416204 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:02.490812 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74aaf94-5c1f-4462-9a49-e62d00f07ee2 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:02.526946 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b713bd36-1329-4f0c-a8a9-65104fe2091d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:02.527681 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713bd36-1329-4f0c-a8a9-65104fe2091d None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:02.527965 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713bd36-1329-4f0c-a8a9-65104fe2091d None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:02.528240 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b713bd36-1329-4f0c-a8a9-65104fe2091d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:02.529420 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713bd36-1329-4f0c-a8a9-65104fe2091d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:02.530874 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b713bd36-1329-4f0c-a8a9-65104fe2091d None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:02.537128 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 508/1010] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:55:02 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:02.538381 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549476e-287a-401a-b44e-7885e56481ee None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:02.643008 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5549476e-287a-401a-b44e-7885e56481ee None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:02.646702 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:02.678839 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad54bd1-7f48-4104-b4e2-e3d8926497ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:02.679538 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad54bd1-7f48-4104-b4e2-e3d8926497ae None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:02.679878 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad54bd1-7f48-4104-b4e2-e3d8926497ae None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:02.680155 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad54bd1-7f48-4104-b4e2-e3d8926497ae None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:02.680920 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad54bd1-7f48-4104-b4e2-e3d8926497ae None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:02.682370 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad54bd1-7f48-4104-b4e2-e3d8926497ae None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:02.716738 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 503/1011] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 23 16:55:02 2022] POST /identity/v3/projects => generated 311 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:55:03.130727 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9a5e4-470f-4f9b-9ff3-4c25623c7cdb None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:03.131187 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9a5e4-470f-4f9b-9ff3-4c25623c7cdb None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:03.131360 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9a5e4-470f-4f9b-9ff3-4c25623c7cdb None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:03.133165 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 509/1012] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:55:03.138341 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7565b13-c199-4ba0-a762-cade850364f8 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:03.138628 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7565b13-c199-4ba0-a762-cade850364f8 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:03.138899 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7565b13-c199-4ba0-a762-cade850364f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:03.156520 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-f7565b13-c199-4ba0-a762-cade850364f8 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:03.157673 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f7565b13-c199-4ba0-a762-cade850364f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:03.226246 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 504/1013] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:03 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 16:55:03.248457 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9a5e4-470f-4f9b-9ff3-4c25623c7cdb None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:03.277300 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1802c3-acec-447f-91c0-3924c425a167 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:03.277822 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1802c3-acec-447f-91c0-3924c425a167 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:03.277822 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1802c3-acec-447f-91c0-3924c425a167 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:03.279426 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 505/1014] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:55:03.284510 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074afa66-1039-446b-8f79-326a8a07c4e3 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:03.284794 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074afa66-1039-446b-8f79-326a8a07c4e3 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:03.285127 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074afa66-1039-446b-8f79-326a8a07c4e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:03.301861 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-074afa66-1039-446b-8f79-326a8a07c4e3 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:03.302888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-074afa66-1039-446b-8f79-326a8a07c4e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:03.361366 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9a5e4-470f-4f9b-9ff3-4c25623c7cdb None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:03.362902 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:03.370973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 506/1015] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:03 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 16:55:03.392006 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074afa66-1039-446b-8f79-326a8a07c4e3 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:03.400341 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d834c2a-0131-4adf-acb0-6cb1b19e8de8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:03.401199 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d834c2a-0131-4adf-acb0-6cb1b19e8de8 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:03.401478 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d834c2a-0131-4adf-acb0-6cb1b19e8de8 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:03.401774 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d834c2a-0131-4adf-acb0-6cb1b19e8de8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:03.402418 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d834c2a-0131-4adf-acb0-6cb1b19e8de8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:03.403811 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d834c2a-0131-4adf-acb0-6cb1b19e8de8 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:03.410661 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:03.416269 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-0d834c2a-0131-4adf-acb0-6cb1b19e8de8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 23 16:55:03.416269 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:03.416269 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:55:03.416269 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:55:03.416269 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:55:03.416269 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:55:03.416269 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:55:03.416269 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:55:03.416269 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 16:55:03.416269 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:55:03.416269 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:55:03.416269 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:55:03.416269 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:03.416269 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:03.416269 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:03.416269 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:03.416269 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:03.417333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:03.417333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:03.417333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:03.417333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:03.417333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:03.417333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:03.417333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:55:03.417333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:55:03.417333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:55:03.417333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:55:03.417333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:03.417333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:03.417333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:03.417333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:03.417333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:03.417333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:03.418219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:03.418219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:55:03.418219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:03.418219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:03.418219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:03.418219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:03.418219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:03.418219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:03.418219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:03.418219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:03.418219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:55:03.418219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:55:03.418219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 16:55:03.418219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:55:03.418219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 510/1016] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:55:03 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:55:03.423748 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d834c2a-0131-4adf-acb0-6cb1b19e8de8 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:03.510398 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074afa66-1039-446b-8f79-326a8a07c4e3 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:03.541005 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d834c2a-0131-4adf-acb0-6cb1b19e8de8 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:03.549182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb06ae-f976-4688-90ee-9704604e8489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:03.550212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb06ae-f976-4688-90ee-9704604e8489 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:03.550497 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb06ae-f976-4688-90ee-9704604e8489 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:03.550771 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb06ae-f976-4688-90ee-9704604e8489 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:03.551532 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb06ae-f976-4688-90ee-9704604e8489 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:03.552923 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52cb06ae-f976-4688-90ee-9704604e8489 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:03.567843 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 507/1017] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 23 16:55:03 2022] POST /identity/v3/services => generated 242 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:55:03.590581 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad39c92-3175-46af-b982-5a1596f27dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:03.591410 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad39c92-3175-46af-b982-5a1596f27dee None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:03.591796 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad39c92-3175-46af-b982-5a1596f27dee None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:03.592139 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad39c92-3175-46af-b982-5a1596f27dee None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:03.592995 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad39c92-3175-46af-b982-5a1596f27dee None admin] RBAC: Adding query filter params (name=compute) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:03.593372 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad39c92-3175-46af-b982-5a1596f27dee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:03.594946 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad39c92-3175-46af-b982-5a1596f27dee None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:03.601560 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 511/1018] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 23 16:55:03 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:55:03.608421 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb06ae-f976-4688-90ee-9704604e8489 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:03.696042 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:03.723623 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52cb06ae-f976-4688-90ee-9704604e8489 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:03.761173 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5428e439-96df-4483-b283-11aff7f0732d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:03.761844 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428e439-96df-4483-b283-11aff7f0732d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:03.762132 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428e439-96df-4483-b283-11aff7f0732d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:03.762406 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5428e439-96df-4483-b283-11aff7f0732d None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:03.763387 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428e439-96df-4483-b283-11aff7f0732d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:03.763554 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428e439-96df-4483-b283-11aff7f0732d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:03.764932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5428e439-96df-4483-b283-11aff7f0732d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:03.770612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 508/1019] 10.222.0.13 () {56 vars in 1112 bytes} [Mon May 23 16:55:03 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:03.776163 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad39c92-3175-46af-b982-5a1596f27dee None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:03.806601 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:03.885205 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad39c92-3175-46af-b982-5a1596f27dee None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:03.921095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd409c0-0291-46d2-819a-2752fabd3cca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:03.921779 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd409c0-0291-46d2-819a-2752fabd3cca None admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:03.922062 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd409c0-0291-46d2-819a-2752fabd3cca None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:03.922330 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd409c0-0291-46d2-819a-2752fabd3cca None admin] PATH_INFO: `/v3/endpoints` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:03.923175 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd409c0-0291-46d2-819a-2752fabd3cca None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:03.924723 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd409c0-0291-46d2-819a-2752fabd3cca None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:03.943945 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 512/1020] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 23 16:55:03 2022] POST /identity/v3/endpoints => generated 334 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:55:04.961508 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030b455-db8e-427d-8c29-7760f98ebf09 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:04.962111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030b455-db8e-427d-8c29-7760f98ebf09 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:04.962111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a030b455-db8e-427d-8c29-7760f98ebf09 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:04.963662 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 509/1021] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:55:04.968805 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3d5e9-34f8-4577-81b4-2853927633df None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:04.969248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3d5e9-34f8-4577-81b4-2853927633df None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:04.969327 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da3d5e9-34f8-4577-81b4-2853927633df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:04.986163 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-9da3d5e9-34f8-4577-81b4-2853927633df None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:04.987367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-9da3d5e9-34f8-4577-81b4-2853927633df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:05.060542 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 513/1022] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:04 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:55:05.084575 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030b455-db8e-427d-8c29-7760f98ebf09 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:05.194670 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a030b455-db8e-427d-8c29-7760f98ebf09 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:05.228344 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb81c2c3-e4bf-4f6a-bdcb-0b8fb83de4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:05.229064 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81c2c3-e4bf-4f6a-bdcb-0b8fb83de4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:05.229347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81c2c3-e4bf-4f6a-bdcb-0b8fb83de4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:05.229620 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb81c2c3-e4bf-4f6a-bdcb-0b8fb83de4a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:05.230787 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81c2c3-e4bf-4f6a-bdcb-0b8fb83de4a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:05.232259 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb81c2c3-e4bf-4f6a-bdcb-0b8fb83de4a0 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:05.238845 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 510/1023] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:55:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:05.240258 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3d5e9-34f8-4577-81b4-2853927633df None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:05.250535 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:05.344733 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da3d5e9-34f8-4577-81b4-2853927633df None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:05.380368 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d9f1cb-f6e7-450e-ab73-1144c9bdeec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:05.381050 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9f1cb-f6e7-450e-ab73-1144c9bdeec7 None admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:05.381335 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9f1cb-f6e7-450e-ab73-1144c9bdeec7 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:05.381612 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d9f1cb-f6e7-450e-ab73-1144c9bdeec7 None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:05.382647 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d9f1cb-f6e7-450e-ab73-1144c9bdeec7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:05.384120 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d9f1cb-f6e7-450e-ab73-1144c9bdeec7 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:05.405114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 514/1024] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 23 16:55:05 2022] POST /identity/v3/users => generated 285 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 16:55:05.812594 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8abfd41-0bc0-4bd4-ba21-aca84b3aa55e None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:05.813001 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8abfd41-0bc0-4bd4-ba21-aca84b3aa55e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:05.813176 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8abfd41-0bc0-4bd4-ba21-aca84b3aa55e None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:05.814648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 511/1025] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:55:05.820121 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeceb42-61ab-4d81-a565-c1904bf0c023 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:05.820374 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeceb42-61ab-4d81-a565-c1904bf0c023 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:05.820661 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdeceb42-61ab-4d81-a565-c1904bf0c023 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:05.837876 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-fdeceb42-61ab-4d81-a565-c1904bf0c023 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:05.839041 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-fdeceb42-61ab-4d81-a565-c1904bf0c023 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:05.901017 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:05.911664 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 515/1026] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:05 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:55:05.934345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8abfd41-0bc0-4bd4-ba21-aca84b3aa55e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:05.946041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:06.043121 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8abfd41-0bc0-4bd4-ba21-aca84b3aa55e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:06.082440 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a5b858a-002c-43a9-876c-02d99262624a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:06.083530 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b858a-002c-43a9-876c-02d99262624a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:06.083951 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b858a-002c-43a9-876c-02d99262624a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:06.084383 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a5b858a-002c-43a9-876c-02d99262624a None admin] PATH_INFO: `/v3/services/image` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:06.085179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5b858a-002c-43a9-876c-02d99262624a None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:06.086704 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a5b858a-002c-43a9-876c-02d99262624a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:06.099218 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-5a5b858a-002c-43a9-876c-02d99262624a None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 23 16:55:06.099218 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:06.099218 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:55:06.099218 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:55:06.099218 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:55:06.099218 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:55:06.099218 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:55:06.099218 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:55:06.099218 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 16:55:06.099218 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:06.099218 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:55:06.099218 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:06.099218 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:06.099218 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:06.099218 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:06.099218 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:06.099218 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:06.100282 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:06.100282 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:06.100282 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:06.100282 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:06.100282 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:06.100282 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:06.100282 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:55:06.100282 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:55:06.100282 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:55:06.100282 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:55:06.100282 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:06.100282 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:06.100282 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:06.100282 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:06.100282 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:06.100282 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:06.101111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:06.101111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:55:06.101111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:06.101111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:06.101111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:06.101111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:06.101111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:06.101111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:06.101111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:06.101111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:06.101111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:55:06.101111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:55:06.101111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 16:55:06.101111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:06.101857 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 512/1027] 10.222.0.13 () {56 vars in 1105 bytes} [Mon May 23 16:55:05 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:55:06.107157 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeceb42-61ab-4d81-a565-c1904bf0c023 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:06.210166 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdeceb42-61ab-4d81-a565-c1904bf0c023 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:06.243832 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717930c3-9430-4167-91ed-3b479ed6dc16 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:06.244136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717930c3-9430-4167-91ed-3b479ed6dc16 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:06.244394 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717930c3-9430-4167-91ed-3b479ed6dc16 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:06.245964 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 513/1028] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:55:06.246067 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1238da-aab6-46fb-bf9c-ef73c52fd6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:06.246611 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1238da-aab6-46fb-bf9c-ef73c52fd6cf None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:06.246942 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1238da-aab6-46fb-bf9c-ef73c52fd6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:06.247222 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1238da-aab6-46fb-bf9c-ef73c52fd6cf None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:06.247947 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1238da-aab6-46fb-bf9c-ef73c52fd6cf None admin] RBAC: Adding query filter params (name=image) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:06.248254 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1238da-aab6-46fb-bf9c-ef73c52fd6cf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:06.249705 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1238da-aab6-46fb-bf9c-ef73c52fd6cf None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:06.251044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229a429-14df-4f12-967b-43e7ab707f5c None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:06.251327 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229a429-14df-4f12-967b-43e7ab707f5c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:06.251961 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229a429-14df-4f12-967b-43e7ab707f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:06.254858 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 516/1029] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:55:06 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:06.261065 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1238da-aab6-46fb-bf9c-ef73c52fd6cf None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:06.270525 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-a229a429-14df-4f12-967b-43e7ab707f5c None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:06.271744 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a229a429-14df-4f12-967b-43e7ab707f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:06.325323 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:06.340316 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 514/1030] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:06 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 16:55:06.364272 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229a429-14df-4f12-967b-43e7ab707f5c None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:06.366394 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:06.371348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1238da-aab6-46fb-bf9c-ef73c52fd6cf None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:06.408919 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63efdd-98ce-47a0-8643-5721242cbe7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:06.409925 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63efdd-98ce-47a0-8643-5721242cbe7b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:06.410441 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63efdd-98ce-47a0-8643-5721242cbe7b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:06.411024 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a63efdd-98ce-47a0-8643-5721242cbe7b None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:06.412112 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63efdd-98ce-47a0-8643-5721242cbe7b None admin] RBAC: Adding query filter params (type=image) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:06.412606 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63efdd-98ce-47a0-8643-5721242cbe7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:06.414369 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a63efdd-98ce-47a0-8643-5721242cbe7b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:06.425089 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 517/1031] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:55:06 2022] GET /identity/v3/services?type=image => generated 350 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:55:06.426105 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63efdd-98ce-47a0-8643-5721242cbe7b None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:06.465162 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229a429-14df-4f12-967b-43e7ab707f5c None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:06.505397 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782dd707-5509-4b81-b1a6-94d32a055810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:06.506992 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782dd707-5509-4b81-b1a6-94d32a055810 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:06.507587 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782dd707-5509-4b81-b1a6-94d32a055810 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:06.507855 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782dd707-5509-4b81-b1a6-94d32a055810 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:06.519469 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782dd707-5509-4b81-b1a6-94d32a055810 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:55:06.527494 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-782dd707-5509-4b81-b1a6-94d32a055810 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 16:55:06.527494 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:06.527494 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:06.527494 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:06.527494 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:06.527494 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:06.527494 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:06.527494 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:06.527494 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:06.527494 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:06.527494 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:06.527494 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:06.527494 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:55:06.527494 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:55:06.527494 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:55:06.527494 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:55:06.528431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:55:06.528431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:55:06.528431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:55:06.528431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:55:06.528431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:06.528431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:06.528431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:06.528431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:06.528431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:06.528431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:06.528431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:06.528431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:55:06.528431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:06.528431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:06.528431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:06.528431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:06.529266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:06.529266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:06.529266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:06.529266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:06.529266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:55:06.529266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:55:06.529266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:55:06.529266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:55:06.529266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:55:06.529266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:55:06.529266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 16:55:06.529266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:06.529266 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 515/1032] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 23 16:55:06 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:55:06.534475 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782dd707-5509-4b81-b1a6-94d32a055810 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:06.540299 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a63efdd-98ce-47a0-8643-5721242cbe7b None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:06.567175 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:06.584976 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d5761-c7a4-47ac-8663-692b3b9f0654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:06.586291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d5761-c7a4-47ac-8663-692b3b9f0654 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:06.587103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d5761-c7a4-47ac-8663-692b3b9f0654 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:06.587600 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478d5761-c7a4-47ac-8663-692b3b9f0654 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:06.589062 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d5761-c7a4-47ac-8663-692b3b9f0654 None admin] RBAC: Adding query filter params (interface=public, service_id=45245eff7d2b45c981481aacead4d07e, region_id=RegionOne) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:06.589593 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d5761-c7a4-47ac-8663-692b3b9f0654 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:06.591581 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478d5761-c7a4-47ac-8663-692b3b9f0654 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:06.601850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 518/1033] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 23 16:55:06 2022] GET /identity/v3/endpoints?service_id=45245eff7d2b45c981481aacead4d07e&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:06.602945 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d5761-c7a4-47ac-8663-692b3b9f0654 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:06.605765 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:06.648230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782dd707-5509-4b81-b1a6-94d32a055810 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:06.682816 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7a96e-3925-43c2-ace5-56db322f79ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:06.683543 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7a96e-3925-43c2-ace5-56db322f79ae None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:06.683824 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7a96e-3925-43c2-ace5-56db322f79ae None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:06.684097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7a96e-3925-43c2-ace5-56db322f79ae None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:06.684847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e7a96e-3925-43c2-ace5-56db322f79ae None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:06.685155 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e7a96e-3925-43c2-ace5-56db322f79ae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:06.686591 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e7a96e-3925-43c2-ace5-56db322f79ae None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:06.695713 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 516/1034] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 23 16:55:06 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:55:06.701350 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7a96e-3925-43c2-ace5-56db322f79ae None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:06.724410 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478d5761-c7a4-47ac-8663-692b3b9f0654 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:06.760916 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fd12a-3d23-4025-8d0e-4a5e75f9bc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:06.762065 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fd12a-3d23-4025-8d0e-4a5e75f9bc9f None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:06.762611 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fd12a-3d23-4025-8d0e-4a5e75f9bc9f None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:06.763085 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301fd12a-3d23-4025-8d0e-4a5e75f9bc9f None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:06.763935 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301fd12a-3d23-4025-8d0e-4a5e75f9bc9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:06.765689 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301fd12a-3d23-4025-8d0e-4a5e75f9bc9f None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:06.772643 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 519/1035] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 23 16:55:06 2022] GET /identity/v3/services => generated 1957 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 16:55:06.809217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e7a96e-3925-43c2-ace5-56db322f79ae None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:06.843483 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:06.857289 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ad889-552e-4bf4-b150-18364ab5a2eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:06.858105 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ad889-552e-4bf4-b150-18364ab5a2eb None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:06.858357 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ad889-552e-4bf4-b150-18364ab5a2eb None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:06.858658 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1ad889-552e-4bf4-b150-18364ab5a2eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:06.871553 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ad889-552e-4bf4-b150-18364ab5a2eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:06.873052 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1ad889-552e-4bf4-b150-18364ab5a2eb None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:06.893382 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-4e1ad889-552e-4bf4-b150-18364ab5a2eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:55:06.893382 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:06.893382 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:06.893382 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:06.893382 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:06.893382 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:06.893382 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:06.893382 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:06.893382 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:06.893382 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:06.893382 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:06.893382 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:06.893382 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:55:06.893382 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:55:06.893382 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:55:06.894400 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:55:06.894400 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:06.894400 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:06.894400 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:06.894400 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:06.894400 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:06.894400 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:06.894400 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:06.894400 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:55:06.894400 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:06.894400 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:06.894400 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:06.894400 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:06.894400 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:06.894400 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:06.894400 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:06.894400 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:06.899878 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:55:06.899878 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:55:06.899878 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:55:06.899878 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:06.899878 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 517/1036] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:55:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:55:06.902105 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fd12a-3d23-4025-8d0e-4a5e75f9bc9f None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:06.963755 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:07.020873 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301fd12a-3d23-4025-8d0e-4a5e75f9bc9f None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:07.056297 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe95c7-7897-4527-a10d-12c891f3f2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:07.056972 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe95c7-7897-4527-a10d-12c891f3f2fd None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:07.057224 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe95c7-7897-4527-a10d-12c891f3f2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:07.057523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59fe95c7-7897-4527-a10d-12c891f3f2fd None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:07.058248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe95c7-7897-4527-a10d-12c891f3f2fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:07.058566 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe95c7-7897-4527-a10d-12c891f3f2fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:07.059983 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59fe95c7-7897-4527-a10d-12c891f3f2fd None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:07.072483 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 520/1037] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:55:07.078181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ad889-552e-4bf4-b150-18364ab5a2eb None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:07.184460 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1ad889-552e-4bf4-b150-18364ab5a2eb None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:07.218722 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f598b4b-5d57-4bc9-8ff1-d7660fc75fbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:07.219568 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f598b4b-5d57-4bc9-8ff1-d7660fc75fbc None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:07.219847 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f598b4b-5d57-4bc9-8ff1-d7660fc75fbc None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:07.220127 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f598b4b-5d57-4bc9-8ff1-d7660fc75fbc None admin] PATH_INFO: `/v3/users/nova` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:07.228840 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f598b4b-5d57-4bc9-8ff1-d7660fc75fbc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:07.230331 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f598b4b-5d57-4bc9-8ff1-d7660fc75fbc None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:07.245856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-0f598b4b-5d57-4bc9-8ff1-d7660fc75fbc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 16:55:07.245856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:07.245856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:07.245856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:07.245856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:07.245856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:07.245856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:07.245856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:07.245856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:07.245856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:07.245856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:07.245856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:07.245856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:55:07.245856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:55:07.245856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:55:07.245856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:55:07.246854 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:07.246854 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:07.246854 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:55:07.246854 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:55:07.246854 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:55:07.246854 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:55:07.246854 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:07.246854 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:07.246854 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:07.246854 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:07.246854 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:07.246854 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:55:07.246854 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:07.246854 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:07.246854 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:07.246854 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:07.246854 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:07.246854 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:07.248182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:07.248182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:07.248182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:55:07.248182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:55:07.248182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:55:07.248182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:55:07.248182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:55:07.248182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:55:07.248182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 16:55:07.248182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:07.250993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 518/1038] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 23 16:55:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:55:07.253595 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe95c7-7897-4527-a10d-12c891f3f2fd None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:07.367464 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59fe95c7-7897-4527-a10d-12c891f3f2fd None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:07.378541 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:07.405154 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9529f8d-0b8e-48a6-9f1c-83d3d7e235ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:07.405828 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9529f8d-0b8e-48a6-9f1c-83d3d7e235ad None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:07.406110 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9529f8d-0b8e-48a6-9f1c-83d3d7e235ad None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:07.406380 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9529f8d-0b8e-48a6-9f1c-83d3d7e235ad None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:07.407246 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9529f8d-0b8e-48a6-9f1c-83d3d7e235ad None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:07.407557 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9529f8d-0b8e-48a6-9f1c-83d3d7e235ad None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:07.409090 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9529f8d-0b8e-48a6-9f1c-83d3d7e235ad None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:07.426607 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 521/1039] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 23 16:55:07 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:07.432454 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f598b4b-5d57-4bc9-8ff1-d7660fc75fbc None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:07.473855 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:07.549015 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f598b4b-5d57-4bc9-8ff1-d7660fc75fbc None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:07.583817 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfebf1-c7dc-4595-81cd-a4c10d8e40fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:07.584471 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfebf1-c7dc-4595-81cd-a4c10d8e40fb None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:07.584753 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfebf1-c7dc-4595-81cd-a4c10d8e40fb None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:07.585021 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ebfebf1-c7dc-4595-81cd-a4c10d8e40fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:07.652491 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfebf1-c7dc-4595-81cd-a4c10d8e40fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:07.653958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ebfebf1-c7dc-4595-81cd-a4c10d8e40fb None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:07.675215 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bf7d0-3371-4544-a1fe-3c532c470fde None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:07.675484 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bf7d0-3371-4544-a1fe-3c532c470fde None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:07.675768 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d04bf7d0-3371-4544-a1fe-3c532c470fde None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:07.677129 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 522/1040] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:55:07.681931 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981036a5-fe21-4afc-9498-52307566cac6 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:07.682230 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981036a5-fe21-4afc-9498-52307566cac6 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:07.682504 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981036a5-fe21-4afc-9498-52307566cac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:07.687865 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:07.700021 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-981036a5-fe21-4afc-9498-52307566cac6 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:07.701188 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-981036a5-fe21-4afc-9498-52307566cac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:07.756482 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 519/1041] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:55:07 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 16:55:07.762401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfebf1-c7dc-4595-81cd-a4c10d8e40fb None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:07.775865 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 523/1042] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:07 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 16:55:07.798119 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981036a5-fe21-4afc-9498-52307566cac6 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:07.872671 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ebfebf1-c7dc-4595-81cd-a4c10d8e40fb None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:07.901727 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:07.907275 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981036a5-fe21-4afc-9498-52307566cac6 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:07.910460 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c702c-8e28-449f-b37c-d73f5f7ad0ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:07.912155 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c702c-8e28-449f-b37c-d73f5f7ad0ce None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:07.912437 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c702c-8e28-449f-b37c-d73f5f7ad0ce None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:07.912708 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c6c702c-8e28-449f-b37c-d73f5f7ad0ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:07.915969 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:07.924961 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c702c-8e28-449f-b37c-d73f5f7ad0ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:07.926618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c6c702c-8e28-449f-b37c-d73f5f7ad0ce None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:07.945981 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-6c6c702c-8e28-449f-b37c-d73f5f7ad0ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:55:07.945981 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:07.945981 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:07.945981 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:07.945981 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:07.945981 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:07.945981 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:07.945981 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:07.945981 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:07.945981 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:07.945981 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:07.945981 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:07.945981 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:55:07.945981 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:55:07.945981 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:55:07.946938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:55:07.946938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:07.946938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:07.946938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:07.946938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:07.946938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:07.946938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:07.946938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:07.946938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:55:07.946938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:07.946938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:07.946938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:07.946938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:07.946938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:07.946938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:07.946938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:07.946938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:07.947849 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:55:07.947849 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:55:07.947849 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:55:07.947849 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:07.947849 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 520/1043] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:55:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:55:07.952003 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb8e6c-d3ea-4438-b4c1-72ab3ec56163 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:07.952657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c702c-8e28-449f-b37c-d73f5f7ad0ce None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:07.952834 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb8e6c-d3ea-4438-b4c1-72ab3ec56163 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:07.953086 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb8e6c-d3ea-4438-b4c1-72ab3ec56163 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:07.953352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ffb8e6c-d3ea-4438-b4c1-72ab3ec56163 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:07.968485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ffb8e6c-d3ea-4438-b4c1-72ab3ec56163 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:55:07.976311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-0ffb8e6c-d3ea-4438-b4c1-72ab3ec56163 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:55:07.976311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:07.976311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:07.976311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:07.976311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:07.976311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:07.976311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:07.976311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:07.976311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:07.976311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:07.976311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:07.976311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:07.976311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:55:07.976311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:55:07.976311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:55:07.976311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:55:07.977272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:55:07.977272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:55:07.977272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:55:07.977272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:55:07.977272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:07.977272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:07.977272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:07.977272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:07.977272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:07.977272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:07.977272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:07.977272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:55:07.977272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:07.977272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:07.977272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:07.977272 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:07.978941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:07.978941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:07.978941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:07.978941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:07.978941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:55:07.978941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:55:07.978941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:55:07.978941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:55:07.978941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:55:07.978941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:55:07.978941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:55:07.978941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:55:07.978941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 524/1044] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:55:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:55:07.983359 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb8e6c-d3ea-4438-b4c1-72ab3ec56163 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:08.064666 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c6c702c-8e28-449f-b37c-d73f5f7ad0ce None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:08.093553 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ffb8e6c-d3ea-4438-b4c1-72ab3ec56163 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:08.099988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6039a2b-606d-4b96-a65f-2b14f2bb9895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:08.100660 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6039a2b-606d-4b96-a65f-2b14f2bb9895 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:08.101066 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6039a2b-606d-4b96-a65f-2b14f2bb9895 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:08.101341 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6039a2b-606d-4b96-a65f-2b14f2bb9895 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:08.102204 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6039a2b-606d-4b96-a65f-2b14f2bb9895 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:08.102516 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6039a2b-606d-4b96-a65f-2b14f2bb9895 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:08.103971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6039a2b-606d-4b96-a65f-2b14f2bb9895 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:08.119967 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 521/1045] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:55:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:55:08.120984 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6039a2b-606d-4b96-a65f-2b14f2bb9895 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:08.134152 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02519a-55df-4107-b91f-ce8fc739ae39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:08.135004 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02519a-55df-4107-b91f-ce8fc739ae39 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:08.135294 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02519a-55df-4107-b91f-ce8fc739ae39 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:08.135572 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc02519a-55df-4107-b91f-ce8fc739ae39 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:08.136322 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc02519a-55df-4107-b91f-ce8fc739ae39 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:08.136635 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc02519a-55df-4107-b91f-ce8fc739ae39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:08.138018 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc02519a-55df-4107-b91f-ce8fc739ae39 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:08.147919 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 525/1046] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:55:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:55:08.153259 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02519a-55df-4107-b91f-ce8fc739ae39 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:08.176080 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:08.179484 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:08.234429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6039a2b-606d-4b96-a65f-2b14f2bb9895 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:08.267432 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc02519a-55df-4107-b91f-ce8fc739ae39 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:08.271151 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df49725f-b682-45a3-93da-2dfd789bf6f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:08.272122 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49725f-b682-45a3-93da-2dfd789bf6f8 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:08.272578 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49725f-b682-45a3-93da-2dfd789bf6f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:08.272998 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df49725f-b682-45a3-93da-2dfd789bf6f8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:08.279968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df49725f-b682-45a3-93da-2dfd789bf6f8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:08.281606 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df49725f-b682-45a3-93da-2dfd789bf6f8 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:08.296479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-df49725f-b682-45a3-93da-2dfd789bf6f8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:55:08.296479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:08.296479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:08.296479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:08.296479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:08.296479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:08.296479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:08.296479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:08.296479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:08.296479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:08.296479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:08.296479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:08.296479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:55:08.296479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:55:08.296479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:55:08.297498 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:55:08.297498 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:08.297498 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:08.297498 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:08.297498 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:08.297498 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:08.297498 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:08.297498 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:08.297498 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:55:08.297498 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:08.297498 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:08.297498 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:08.297498 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:08.297498 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:08.297498 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:08.297498 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:08.297498 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:08.298698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:55:08.298698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:55:08.298698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:55:08.298698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:55:08.298698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:55:08.298698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:55:08.298698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:55:08.298698 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:08.300759 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 522/1047] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:55:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:55:08.306363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df49725f-b682-45a3-93da-2dfd789bf6f8 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:08.307378 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a0408-992c-4369-be25-942ea7c1f197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:08.308260 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a0408-992c-4369-be25-942ea7c1f197 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:08.308539 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a0408-992c-4369-be25-942ea7c1f197 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:08.308784 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564a0408-992c-4369-be25-942ea7c1f197 None admin] PATH_INFO: `/v3/users/1a76292e679a48799d615853a42c8e04` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:08.324087 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a0408-992c-4369-be25-942ea7c1f197 None admin] RBAC: Authorizing `identity:get_user(user_id=1a76292e679a48799d615853a42c8e04)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:08.325472 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564a0408-992c-4369-be25-942ea7c1f197 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:08.327348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 526/1048] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:55:08 2022] GET /identity/v3/users/1a76292e679a48799d615853a42c8e04 => generated 285 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:08.332946 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a0408-992c-4369-be25-942ea7c1f197 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:08.411183 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df49725f-b682-45a3-93da-2dfd789bf6f8 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:08.416006 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:08.422162 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:08.440399 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564a0408-992c-4369-be25-942ea7c1f197 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:08.449781 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8522a32-a686-49ba-b0df-c089f8b52f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:08.450807 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8522a32-a686-49ba-b0df-c089f8b52f05 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:08.451191 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8522a32-a686-49ba-b0df-c089f8b52f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:08.451763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8522a32-a686-49ba-b0df-c089f8b52f05 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:08.452585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8522a32-a686-49ba-b0df-c089f8b52f05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:08.452876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8522a32-a686-49ba-b0df-c089f8b52f05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:08.454310 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8522a32-a686-49ba-b0df-c089f8b52f05 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:08.470314 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 523/1049] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:55:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:55:08.476894 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8522a32-a686-49ba-b0df-c089f8b52f05 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:08.486644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bf782-dfd0-4ab7-a6d2-30f4cb2bc145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:08.487325 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bf782-dfd0-4ab7-a6d2-30f4cb2bc145 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:08.487325 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bf782-dfd0-4ab7-a6d2-30f4cb2bc145 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:08.487325 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8bf782-dfd0-4ab7-a6d2-30f4cb2bc145 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:08.565104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8bf782-dfd0-4ab7-a6d2-30f4cb2bc145 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:08.566665 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8bf782-dfd0-4ab7-a6d2-30f4cb2bc145 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:08.582508 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8522a32-a686-49ba-b0df-c089f8b52f05 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:08.619994 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14bb471-9524-4f42-8775-7d12b467f0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:08.620657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14bb471-9524-4f42-8775-7d12b467f0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:08.620955 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14bb471-9524-4f42-8775-7d12b467f0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:08.621213 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14bb471-9524-4f42-8775-7d12b467f0b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:08.622033 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14bb471-9524-4f42-8775-7d12b467f0b3 None admin] RBAC: Adding query filter params (role.id=7141f022bbca499c80cc11cca858e064, scope.project.id=0595667fb2e04a3ca4716f669d6698b0, user.id=225c7b385f12445fab86b3b5606b1da1) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:08.622341 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14bb471-9524-4f42-8775-7d12b467f0b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:08.623796 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14bb471-9524-4f42-8775-7d12b467f0b3 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:08.629782 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 524/1050] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:55:08 2022] GET /identity/v3/role_assignments?user.id=225c7b385f12445fab86b3b5606b1da1&scope.project.id=0595667fb2e04a3ca4716f669d6698b0&role.id=7141f022bbca499c80cc11cca858e064 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:55:08.668622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:08.702761 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 527/1051] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:55:08 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 16:55:08.708739 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14bb471-9524-4f42-8775-7d12b467f0b3 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:08.746305 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:08.835495 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14bb471-9524-4f42-8775-7d12b467f0b3 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:08.870383 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215c4a04-4cbd-40c6-b3cd-b1c3f158aaa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:08.871588 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c4a04-4cbd-40c6-b3cd-b1c3f158aaa6 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:08.871871 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c4a04-4cbd-40c6-b3cd-b1c3f158aaa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:08.872144 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215c4a04-4cbd-40c6-b3cd-b1c3f158aaa6 None admin] PATH_INFO: `/v3/projects/163424ef91fc4893889ba520ee61a62d` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:08.873317 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c4a04-4cbd-40c6-b3cd-b1c3f158aaa6 None admin] RBAC: Authorizing `identity:get_project(project_id=163424ef91fc4893889ba520ee61a62d)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:08.874765 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215c4a04-4cbd-40c6-b3cd-b1c3f158aaa6 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:08.877404 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 525/1052] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:55:08 2022] GET /identity/v3/projects/163424ef91fc4893889ba520ee61a62d => generated 311 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:08.883229 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bf782-dfd0-4ab7-a6d2-30f4cb2bc145 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:08.988933 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8bf782-dfd0-4ab7-a6d2-30f4cb2bc145 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:08.998355 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f77b4-47cb-463e-9ba3-062a78ff190c None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:08.998631 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f77b4-47cb-463e-9ba3-062a78ff190c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:08.998939 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-670f77b4-47cb-463e-9ba3-062a78ff190c None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:09.002963 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 526/1053] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:55:09.005699 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1cd36-bdd0-4fd4-bae8-2ba8ccf547d2 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:09.005990 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1cd36-bdd0-4fd4-bae8-2ba8ccf547d2 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:09.006280 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da1cd36-bdd0-4fd4-bae8-2ba8ccf547d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:09.024024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-6da1cd36-bdd0-4fd4-bae8-2ba8ccf547d2 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:09.025611 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-6da1cd36-bdd0-4fd4-bae8-2ba8ccf547d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:09.030171 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1bf129-7d66-45fa-93e7-668fdab28ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:09.030842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bf129-7d66-45fa-93e7-668fdab28ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:09.031164 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bf129-7d66-45fa-93e7-668fdab28ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:09.031440 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1bf129-7d66-45fa-93e7-668fdab28ed7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:09.032269 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1bf129-7d66-45fa-93e7-668fdab28ed7 None admin] RBAC: Adding query filter params (role.id=17bbe7cc1c3641509de5e9dca8534102, scope.project.id=163424ef91fc4893889ba520ee61a62d, user.id=1a76292e679a48799d615853a42c8e04) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:09.032774 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1bf129-7d66-45fa-93e7-668fdab28ed7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:09.034215 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1bf129-7d66-45fa-93e7-668fdab28ed7 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:09.039814 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 528/1054] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:55:08 2022] GET /identity/v3/role_assignments?user.id=1a76292e679a48799d615853a42c8e04&scope.project.id=163424ef91fc4893889ba520ee61a62d&role.id=17bbe7cc1c3641509de5e9dca8534102 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:09.104328 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 527/1055] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:09 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:55:09.127893 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1bf129-7d66-45fa-93e7-668fdab28ed7 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:09.177847 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:09.242284 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1bf129-7d66-45fa-93e7-668fdab28ed7 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:09.277517 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ba579-62cd-489f-9358-4dfa587a057b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:09.278313 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ba579-62cd-489f-9358-4dfa587a057b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:09.278594 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ba579-62cd-489f-9358-4dfa587a057b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:09.278870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632ba579-62cd-489f-9358-4dfa587a057b None admin] PATH_INFO: `/v3/services/image` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:09.279550 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ba579-62cd-489f-9358-4dfa587a057b None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:09.280920 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632ba579-62cd-489f-9358-4dfa587a057b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:09.292579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-632ba579-62cd-489f-9358-4dfa587a057b None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 23 16:55:09.292579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:09.292579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:55:09.292579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:55:09.292579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:55:09.292579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:55:09.292579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:55:09.292579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:55:09.292579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 16:55:09.292579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:55:09.292579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:55:09.292579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:55:09.292579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:09.292579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:09.292579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:09.292579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:09.292579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:09.293648 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:09.293648 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:09.293648 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:09.293648 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:09.293648 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:09.293648 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:09.293648 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:55:09.293648 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:55:09.293648 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:55:09.293648 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:55:09.293648 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:09.293648 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:09.293648 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:09.293648 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:09.293648 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:09.293648 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:09.294481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:09.294481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:55:09.294481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:09.294481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:09.294481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:09.294481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:09.294481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:09.294481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:09.294481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:09.294481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:09.294481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:55:09.294481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:55:09.294481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 16:55:09.294481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:55:09.294481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 529/1056] 10.222.0.13 () {56 vars in 1105 bytes} [Mon May 23 16:55:09 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:55:09.300582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1cd36-bdd0-4fd4-bae8-2ba8ccf547d2 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:09.332382 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:09.409035 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da1cd36-bdd0-4fd4-bae8-2ba8ccf547d2 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:09.443429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e4530-2dde-41a7-97b9-53f6f0c2f1d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:09.444091 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e4530-2dde-41a7-97b9-53f6f0c2f1d9 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:09.444366 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e4530-2dde-41a7-97b9-53f6f0c2f1d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:09.444632 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464e4530-2dde-41a7-97b9-53f6f0c2f1d9 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:09.445328 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e4530-2dde-41a7-97b9-53f6f0c2f1d9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:09.445625 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e4530-2dde-41a7-97b9-53f6f0c2f1d9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:09.447127 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-464e4530-2dde-41a7-97b9-53f6f0c2f1d9 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:09.457183 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 528/1057] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:55:09 2022] GET /identity/v3/services?name=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:09.458412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ba579-62cd-489f-9358-4dfa587a057b None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:09.571261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632ba579-62cd-489f-9358-4dfa587a057b None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:09.578465 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:09.607455 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba13f4b-9184-4f86-9d01-18e444415669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:09.608110 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba13f4b-9184-4f86-9d01-18e444415669 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:09.608392 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba13f4b-9184-4f86-9d01-18e444415669 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:09.608660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba13f4b-9184-4f86-9d01-18e444415669 None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:09.609358 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba13f4b-9184-4f86-9d01-18e444415669 None admin] RBAC: Adding query filter params (type=image) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:09.609661 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba13f4b-9184-4f86-9d01-18e444415669 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:09.611087 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba13f4b-9184-4f86-9d01-18e444415669 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:09.619329 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 530/1058] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:55:09 2022] GET /identity/v3/services?type=image => generated 350 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:55:09.621908 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e4530-2dde-41a7-97b9-53f6f0c2f1d9 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:09.723228 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:09.732264 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464e4530-2dde-41a7-97b9-53f6f0c2f1d9 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:09.775325 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da33e88-cb40-4fb1-a105-f67e13fe9c8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:09.775996 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da33e88-cb40-4fb1-a105-f67e13fe9c8b None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:09.776274 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da33e88-cb40-4fb1-a105-f67e13fe9c8b None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:09.776544 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da33e88-cb40-4fb1-a105-f67e13fe9c8b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:09.777274 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da33e88-cb40-4fb1-a105-f67e13fe9c8b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:09.778641 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da33e88-cb40-4fb1-a105-f67e13fe9c8b None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:09.807160 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 529/1059] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 23 16:55:09 2022] POST /identity/v3/endpoints => generated 327 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:55:10.814263 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498d490-5e01-44dc-aa29-bcdc225b5486 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:10.814795 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498d490-5e01-44dc-aa29-bcdc225b5486 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:10.814795 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0498d490-5e01-44dc-aa29-bcdc225b5486 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:10.816080 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 531/1060] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:55:10.821669 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cc655-a3d9-4489-b9b8-d7427afc25d9 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:10.822089 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cc655-a3d9-4489-b9b8-d7427afc25d9 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:10.822281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833cc655-a3d9-4489-b9b8-d7427afc25d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:10.840021 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-833cc655-a3d9-4489-b9b8-d7427afc25d9 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:10.841234 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-833cc655-a3d9-4489-b9b8-d7427afc25d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:10.926957 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 530/1061] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:55:10.949276 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498d490-5e01-44dc-aa29-bcdc225b5486 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:11.059311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0498d490-5e01-44dc-aa29-bcdc225b5486 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:11.093990 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3142a57-0d5e-4a8f-97ab-de9eba8e5f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:11.094806 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3142a57-0d5e-4a8f-97ab-de9eba8e5f97 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:11.095259 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3142a57-0d5e-4a8f-97ab-de9eba8e5f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:11.095811 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3142a57-0d5e-4a8f-97ab-de9eba8e5f97 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:11.105809 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3142a57-0d5e-4a8f-97ab-de9eba8e5f97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:55:11.113954 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-e3142a57-0d5e-4a8f-97ab-de9eba8e5f97 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 16:55:11.113954 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:11.113954 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:11.113954 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:11.113954 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:11.113954 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:11.113954 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:11.113954 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:11.113954 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:11.113954 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:11.113954 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:11.113954 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:11.113954 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:55:11.113954 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:55:11.113954 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:55:11.113954 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:55:11.115177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:55:11.115177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:55:11.115177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:55:11.115177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:55:11.115177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:11.115177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:11.115177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:11.115177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:11.115177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:11.115177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:11.115177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:11.115177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:55:11.115177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:11.115177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:11.115177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:11.115177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:11.116127 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:11.116127 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:11.116127 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:11.116127 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:11.116127 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:55:11.116127 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:55:11.116127 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:55:11.116127 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:55:11.116127 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:55:11.116127 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:55:11.116127 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 16:55:11.116127 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:55:11.119310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 532/1062] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 23 16:55:10 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:55:11.121107 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cc655-a3d9-4489-b9b8-d7427afc25d9 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:11.198332 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:11.215892 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39607b-ba88-4e27-a3f2-0c55e479f3a4 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:11.216177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39607b-ba88-4e27-a3f2-0c55e479f3a4 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:11.216434 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b39607b-ba88-4e27-a3f2-0c55e479f3a4 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:11.217810 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 533/1063] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:55:11.223036 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990019b-9222-4d51-b9a7-fb5f56265d98 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:11.223310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990019b-9222-4d51-b9a7-fb5f56265d98 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:11.223584 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a990019b-9222-4d51-b9a7-fb5f56265d98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:11.226109 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833cc655-a3d9-4489-b9b8-d7427afc25d9 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:11.240580 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-a990019b-9222-4d51-b9a7-fb5f56265d98 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:11.242429 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-a990019b-9222-4d51-b9a7-fb5f56265d98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:11.262712 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6009ec5-8304-40bc-a673-600687a1c474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:11.263461 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6009ec5-8304-40bc-a673-600687a1c474 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:11.263738 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6009ec5-8304-40bc-a673-600687a1c474 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:11.264005 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6009ec5-8304-40bc-a673-600687a1c474 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:11.264739 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6009ec5-8304-40bc-a673-600687a1c474 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:11.265047 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6009ec5-8304-40bc-a673-600687a1c474 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:11.266460 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6009ec5-8304-40bc-a673-600687a1c474 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:11.270264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:11.275069 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 531/1064] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 23 16:55:11 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:11.280627 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6009ec5-8304-40bc-a673-600687a1c474 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:11.323318 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 534/1065] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 16:55:11.341514 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990019b-9222-4d51-b9a7-fb5f56265d98 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:11.386127 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6009ec5-8304-40bc-a673-600687a1c474 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:11.420972 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cec83c-dce2-4640-ac9c-054adcd41619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:11.421737 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cec83c-dce2-4640-ac9c-054adcd41619 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:11.422018 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cec83c-dce2-4640-ac9c-054adcd41619 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:11.422291 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cec83c-dce2-4640-ac9c-054adcd41619 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:11.430539 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:11.436177 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cec83c-dce2-4640-ac9c-054adcd41619 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:11.437639 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cec83c-dce2-4640-ac9c-054adcd41619 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:11.447234 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a990019b-9222-4d51-b9a7-fb5f56265d98 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:11.455726 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-04cec83c-dce2-4640-ac9c-054adcd41619 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:55:11.455726 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:11.455726 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:11.455726 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:11.455726 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:11.455726 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:11.455726 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:11.455726 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:11.455726 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:11.455726 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:11.455726 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:11.455726 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:11.455726 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:55:11.455726 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:55:11.455726 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:55:11.456616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:55:11.456616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:11.456616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:11.456616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:11.456616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:11.456616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:11.456616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:11.456616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:11.456616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:55:11.456616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:11.456616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:11.456616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:11.456616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:11.456616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:11.456616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:11.456616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:11.456616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:11.457514 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:55:11.457514 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:55:11.457514 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:55:11.457514 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:11.457514 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 532/1066] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:55:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:55:11.462501 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cec83c-dce2-4640-ac9c-054adcd41619 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:11.492079 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf462c-8c42-42e9-a9cc-724d5eb57e8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:11.492997 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf462c-8c42-42e9-a9cc-724d5eb57e8a None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:11.493288 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf462c-8c42-42e9-a9cc-724d5eb57e8a None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:11.493563 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bf462c-8c42-42e9-a9cc-724d5eb57e8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:11.505460 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21bf462c-8c42-42e9-a9cc-724d5eb57e8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:55:11.514035 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-21bf462c-8c42-42e9-a9cc-724d5eb57e8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:55:11.514035 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:11.514035 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:11.514035 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:11.514035 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:11.514035 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:11.514035 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:11.514035 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:11.514035 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:11.514035 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:11.514035 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:11.514035 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:11.514035 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:55:11.514035 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:55:11.514035 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:55:11.514035 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:55:11.514993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:55:11.514993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:55:11.514993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:55:11.514993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:55:11.514993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:11.514993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:11.514993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:11.514993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:11.514993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:11.514993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:11.514993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:11.514993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:55:11.514993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:11.514993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:11.514993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:11.514993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:11.515846 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:11.515846 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:11.515846 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:11.515846 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:11.515846 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:55:11.515846 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:55:11.515846 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:55:11.515846 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:55:11.515846 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:55:11.515846 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:55:11.515846 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:55:11.515846 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:55:11.515846 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 535/1067] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:55:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:55:11.521630 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf462c-8c42-42e9-a9cc-724d5eb57e8a None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:11.546001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:11.568340 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cec83c-dce2-4640-ac9c-054adcd41619 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:11.602024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe06bd1-36ed-42de-a7d2-a8092ac2f4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:11.602694 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe06bd1-36ed-42de-a7d2-a8092ac2f4dc None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:11.603004 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe06bd1-36ed-42de-a7d2-a8092ac2f4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:11.603284 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe06bd1-36ed-42de-a7d2-a8092ac2f4dc None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:11.604009 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe06bd1-36ed-42de-a7d2-a8092ac2f4dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:11.604313 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe06bd1-36ed-42de-a7d2-a8092ac2f4dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:11.605700 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe06bd1-36ed-42de-a7d2-a8092ac2f4dc None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:11.622041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 533/1068] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:55:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:55:11.622928 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe06bd1-36ed-42de-a7d2-a8092ac2f4dc None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:11.630504 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21bf462c-8c42-42e9-a9cc-724d5eb57e8a None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:11.667269 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913328a-8c14-428e-8e92-e429c959feef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:11.668179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913328a-8c14-428e-8e92-e429c959feef None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:11.668720 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913328a-8c14-428e-8e92-e429c959feef None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:11.669162 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a913328a-8c14-428e-8e92-e429c959feef None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:11.670147 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913328a-8c14-428e-8e92-e429c959feef None admin] RBAC: Adding query filter params (name=admin) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:11.670616 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913328a-8c14-428e-8e92-e429c959feef None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:11.672315 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a913328a-8c14-428e-8e92-e429c959feef None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:11.672800 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:11.683622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 536/1069] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:55:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:11.689733 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913328a-8c14-428e-8e92-e429c959feef None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:11.733195 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe06bd1-36ed-42de-a7d2-a8092ac2f4dc None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:11.770174 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3fdda-1b05-4e1d-88a8-704b63dccbc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:11.771111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3fdda-1b05-4e1d-88a8-704b63dccbc6 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:11.771392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3fdda-1b05-4e1d-88a8-704b63dccbc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:11.771770 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3fdda-1b05-4e1d-88a8-704b63dccbc6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:11.772924 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:11.781382 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3fdda-1b05-4e1d-88a8-704b63dccbc6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:11.782933 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3fdda-1b05-4e1d-88a8-704b63dccbc6 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:11.792551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a913328a-8c14-428e-8e92-e429c959feef None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:11.798296 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-21e3fdda-1b05-4e1d-88a8-704b63dccbc6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 16:55:11.798296 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:11.798296 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:11.798296 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:11.798296 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:11.798296 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:11.798296 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:11.798296 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:11.798296 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:11.798296 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:11.798296 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:11.798296 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:11.798296 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:55:11.798296 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:55:11.798296 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:55:11.798296 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:55:11.799304 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:11.799304 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:11.799304 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:55:11.799304 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:55:11.799304 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:55:11.799304 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:55:11.799304 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:11.799304 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:11.799304 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:11.799304 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:11.799304 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:11.799304 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:55:11.799304 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:11.799304 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:11.799304 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:11.799304 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:11.799304 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:11.799304 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:11.800269 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:11.800269 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:11.800269 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:55:11.800269 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:55:11.800269 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:55:11.800269 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:55:11.800269 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:55:11.800269 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:55:11.800269 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 16:55:11.800269 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:11.800269 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 534/1070] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 23 16:55:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:55:11.805160 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3fdda-1b05-4e1d-88a8-704b63dccbc6 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:11.833682 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d2db9-3342-4111-a921-a61c48254175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:11.835358 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d2db9-3342-4111-a921-a61c48254175 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:11.835795 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d2db9-3342-4111-a921-a61c48254175 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:11.836219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9d2db9-3342-4111-a921-a61c48254175 None admin] PATH_INFO: `/v3/users/1a76292e679a48799d615853a42c8e04` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:11.837987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9d2db9-3342-4111-a921-a61c48254175 None admin] RBAC: Authorizing `identity:get_user(user_id=1a76292e679a48799d615853a42c8e04)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:11.839529 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9d2db9-3342-4111-a921-a61c48254175 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:11.846335 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 537/1071] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:55:11 2022] GET /identity/v3/users/1a76292e679a48799d615853a42c8e04 => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:55:11.847659 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d2db9-3342-4111-a921-a61c48254175 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:11.914996 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3fdda-1b05-4e1d-88a8-704b63dccbc6 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:11.926327 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:11.950888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ef243-afd9-442a-8b70-c9881e208c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:11.951657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ef243-afd9-442a-8b70-c9881e208c49 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:11.951944 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ef243-afd9-442a-8b70-c9881e208c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:11.952216 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875ef243-afd9-442a-8b70-c9881e208c49 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:11.953055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ef243-afd9-442a-8b70-c9881e208c49 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:11.953368 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ef243-afd9-442a-8b70-c9881e208c49 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:11.954845 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875ef243-afd9-442a-8b70-c9881e208c49 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:11.961887 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9d2db9-3342-4111-a921-a61c48254175 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:11.977298 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 535/1072] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 23 16:55:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:55:11.978358 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ef243-afd9-442a-8b70-c9881e208c49 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:12.003633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a38f2e-9a61-4456-a960-32583e7e9446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:12.004944 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a38f2e-9a61-4456-a960-32583e7e9446 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:12.005361 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a38f2e-9a61-4456-a960-32583e7e9446 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:12.005641 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a38f2e-9a61-4456-a960-32583e7e9446 None admin] PATH_INFO: `/v3/projects/163424ef91fc4893889ba520ee61a62d` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:12.007622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a38f2e-9a61-4456-a960-32583e7e9446 None admin] RBAC: Authorizing `identity:get_project(project_id=163424ef91fc4893889ba520ee61a62d)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:12.009500 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a38f2e-9a61-4456-a960-32583e7e9446 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:12.016566 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 538/1073] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:55:11 2022] GET /identity/v3/projects/163424ef91fc4893889ba520ee61a62d => generated 311 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:12.017750 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a38f2e-9a61-4456-a960-32583e7e9446 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:12.037880 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:12.080197 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875ef243-afd9-442a-8b70-c9881e208c49 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:12.117215 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a38f2e-9a61-4456-a960-32583e7e9446 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:12.117381 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926e30b-752a-43ef-a07e-503bfcf6a9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:12.118039 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926e30b-752a-43ef-a07e-503bfcf6a9c2 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:12.118317 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926e30b-752a-43ef-a07e-503bfcf6a9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:12.118588 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926e30b-752a-43ef-a07e-503bfcf6a9c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:12.129174 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926e30b-752a-43ef-a07e-503bfcf6a9c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:12.130634 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926e30b-752a-43ef-a07e-503bfcf6a9c2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:12.149024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-b926e30b-752a-43ef-a07e-503bfcf6a9c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:55:12.149024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:12.149024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:12.149024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:12.149024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:12.149024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:12.149024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:12.149024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:12.149024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:12.149024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:12.149024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:12.149024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:12.149024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:55:12.149024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:55:12.149024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:55:12.149927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:55:12.149927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:12.149927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:12.149927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:12.149927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:12.149927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:12.149927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:12.149927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:12.149927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:55:12.149927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:12.149927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:12.149927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:12.149927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:12.149927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:12.149927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:12.149927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:12.149927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:12.150819 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:55:12.150819 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:55:12.150819 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:55:12.150819 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:12.150819 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 536/1074] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:55:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:55:12.151449 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8606b-09d0-4a65-b5b7-73d7aaebaa7f None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:12.151713 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8606b-09d0-4a65-b5b7-73d7aaebaa7f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:12.151988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e8606b-09d0-4a65-b5b7-73d7aaebaa7f None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:12.153259 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 537/1075] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:12 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:55:12.155786 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8606b-09d0-4a65-b5b7-73d7aaebaa7f None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:12.156586 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae6b3d3-3dea-4d33-bd4c-a48cafce1365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:12.157486 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6b3d3-3dea-4d33-bd4c-a48cafce1365 None admin] REQUEST_METHOD: `PUT` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:12.157764 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6b3d3-3dea-4d33-bd4c-a48cafce1365 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:12.158041 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae6b3d3-3dea-4d33-bd4c-a48cafce1365 None admin] PATH_INFO: `/v3/projects/163424ef91fc4893889ba520ee61a62d/users/1a76292e679a48799d615853a42c8e04/roles/17bbe7cc1c3641509de5e9dca8534102` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:12.160105 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae6b3d3-3dea-4d33-bd4c-a48cafce1365 None admin] RBAC: Authorizing `identity:create_grant(project_id=163424ef91fc4893889ba520ee61a62d, user_id=1a76292e679a48799d615853a42c8e04, role_id=17bbe7cc1c3641509de5e9dca8534102)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:12.161525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae6b3d3-3dea-4d33-bd4c-a48cafce1365 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:12.175822 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 539/1076] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:55:12 2022] PUT /identity/v3/projects/163424ef91fc4893889ba520ee61a62d/users/1a76292e679a48799d615853a42c8e04/roles/17bbe7cc1c3641509de5e9dca8534102 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 16:55:12.177087 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1970c00-fd1c-435a-a70a-7dd6eac08ff4 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:12.177319 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1970c00-fd1c-435a-a70a-7dd6eac08ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:12.177591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1970c00-fd1c-435a-a70a-7dd6eac08ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:12.178471 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:12.215923 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-e1970c00-fd1c-435a-a70a-7dd6eac08ff4 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:12.217645 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-e1970c00-fd1c-435a-a70a-7dd6eac08ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:12.280296 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e8606b-09d0-4a65-b5b7-73d7aaebaa7f None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:12.302726 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 540/1077] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:55:12.316971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d61741-1a1b-4d4b-8f0e-6c937dc8fd69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:12.317654 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d61741-1a1b-4d4b-8f0e-6c937dc8fd69 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:12.317937 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d61741-1a1b-4d4b-8f0e-6c937dc8fd69 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:12.318207 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d61741-1a1b-4d4b-8f0e-6c937dc8fd69 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:12.318965 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d61741-1a1b-4d4b-8f0e-6c937dc8fd69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:12.319279 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d61741-1a1b-4d4b-8f0e-6c937dc8fd69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:12.320711 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d61741-1a1b-4d4b-8f0e-6c937dc8fd69 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:12.324625 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1970c00-fd1c-435a-a70a-7dd6eac08ff4 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:12.338760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 538/1078] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:55:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:12.338989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:12.340361 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d61741-1a1b-4d4b-8f0e-6c937dc8fd69 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:12.430501 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:12.446043 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1970c00-fd1c-435a-a70a-7dd6eac08ff4 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:12.451130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d61741-1a1b-4d4b-8f0e-6c937dc8fd69 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:12.489632 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e8c72-b67d-4aca-b034-e0d2ed7af635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:12.490298 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e8c72-b67d-4aca-b034-e0d2ed7af635 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:12.490631 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e8c72-b67d-4aca-b034-e0d2ed7af635 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:12.490905 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff313ca-f6f0-4472-b6b5-cee6833f1ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:12.491030 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d9e8c72-b67d-4aca-b034-e0d2ed7af635 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:12.491904 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff313ca-f6f0-4472-b6b5-cee6833f1ffb None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:12.492187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff313ca-f6f0-4472-b6b5-cee6833f1ffb None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:12.492458 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff313ca-f6f0-4472-b6b5-cee6833f1ffb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:12.504729 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff313ca-f6f0-4472-b6b5-cee6833f1ffb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:12.506771 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff313ca-f6f0-4472-b6b5-cee6833f1ffb None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:12.526530 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-3ff313ca-f6f0-4472-b6b5-cee6833f1ffb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:55:12.526530 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:12.526530 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:12.526530 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:12.526530 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:12.526530 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:12.526530 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:12.526530 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:12.526530 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:12.526530 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:12.526530 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:12.526530 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:12.526530 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:55:12.526530 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:55:12.526530 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:55:12.527705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:55:12.527705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:12.527705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:12.527705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:12.527705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:12.527705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:12.527705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:12.527705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:12.527705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:55:12.527705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:12.527705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:12.527705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:12.527705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:12.527705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:12.527705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:12.527705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:12.527705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:12.528601 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:55:12.528601 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:55:12.528601 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:55:12.528601 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:55:12.528601 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:55:12.528601 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:55:12.528601 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:55:12.528601 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:12.528601 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 539/1079] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:55:12 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:55:12.534905 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff313ca-f6f0-4472-b6b5-cee6833f1ffb None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:12.561641 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e8c72-b67d-4aca-b034-e0d2ed7af635 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:12.563131 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d9e8c72-b67d-4aca-b034-e0d2ed7af635 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:12.572971 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:12.645227 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff313ca-f6f0-4472-b6b5-cee6833f1ffb None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:12.670954 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 541/1080] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:55:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 16:55:12.674407 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e8c72-b67d-4aca-b034-e0d2ed7af635 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:12.681926 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2170662-f2c9-4fe5-8d06-fb4bcb35f390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:12.682633 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2170662-f2c9-4fe5-8d06-fb4bcb35f390 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:12.682904 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2170662-f2c9-4fe5-8d06-fb4bcb35f390 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:12.683199 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2170662-f2c9-4fe5-8d06-fb4bcb35f390 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:12.683977 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2170662-f2c9-4fe5-8d06-fb4bcb35f390 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:12.684396 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2170662-f2c9-4fe5-8d06-fb4bcb35f390 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:12.685949 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2170662-f2c9-4fe5-8d06-fb4bcb35f390 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:12.695298 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:12.704154 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 540/1081] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:55:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:55:12.704920 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2170662-f2c9-4fe5-8d06-fb4bcb35f390 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:12.779400 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d9e8c72-b67d-4aca-b034-e0d2ed7af635 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:12.789319 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:12.808692 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2170662-f2c9-4fe5-8d06-fb4bcb35f390 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:12.820738 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065bd1a8-aa9f-43cd-b5ca-f811338339a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:12.822030 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065bd1a8-aa9f-43cd-b5ca-f811338339a4 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:12.822828 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065bd1a8-aa9f-43cd-b5ca-f811338339a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:12.823289 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065bd1a8-aa9f-43cd-b5ca-f811338339a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:12.836918 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065bd1a8-aa9f-43cd-b5ca-f811338339a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:12.838558 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065bd1a8-aa9f-43cd-b5ca-f811338339a4 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:12.843567 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf957d9-f43f-4536-9f5b-980a213ef0a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:12.844460 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf957d9-f43f-4536-9f5b-980a213ef0a7 None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:12.844742 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf957d9-f43f-4536-9f5b-980a213ef0a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:12.845049 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf957d9-f43f-4536-9f5b-980a213ef0a7 None admin] PATH_INFO: `/v3/projects/0595667fb2e04a3ca4716f669d6698b0/users/225c7b385f12445fab86b3b5606b1da1/roles/7141f022bbca499c80cc11cca858e064` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:12.847321 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf957d9-f43f-4536-9f5b-980a213ef0a7 None admin] RBAC: Authorizing `identity:create_grant(project_id=0595667fb2e04a3ca4716f669d6698b0, user_id=225c7b385f12445fab86b3b5606b1da1, role_id=7141f022bbca499c80cc11cca858e064)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:12.848741 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf957d9-f43f-4536-9f5b-980a213ef0a7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:12.857078 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-065bd1a8-aa9f-43cd-b5ca-f811338339a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:55:12.857078 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:12.857078 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:12.857078 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:12.857078 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:12.857078 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:12.857078 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:12.857078 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:12.857078 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:12.857078 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:12.857078 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:12.857078 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:12.857078 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:55:12.857078 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:55:12.857078 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:55:12.858137 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:55:12.858137 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:12.858137 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:12.858137 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:12.858137 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:12.858137 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:12.858137 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:12.858137 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:12.858137 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:55:12.858137 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:12.858137 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:12.858137 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:12.858137 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:12.858137 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:12.858137 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:12.858137 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:12.858137 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:12.859186 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:55:12.859186 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:55:12.859186 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:55:12.859186 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:55:12.861220 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 542/1082] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:55:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:55:12.863408 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 541/1083] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:55:12 2022] PUT /identity/v3/projects/0595667fb2e04a3ca4716f669d6698b0/users/225c7b385f12445fab86b3b5606b1da1/roles/7141f022bbca499c80cc11cca858e064 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:55:12.866793 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065bd1a8-aa9f-43cd-b5ca-f811338339a4 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:12.986435 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065bd1a8-aa9f-43cd-b5ca-f811338339a4 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:13.026757 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2b4f94-881d-4dd4-8d40-7081a0b90f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:13.027513 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b4f94-881d-4dd4-8d40-7081a0b90f96 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:13.027795 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b4f94-881d-4dd4-8d40-7081a0b90f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:13.028068 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2b4f94-881d-4dd4-8d40-7081a0b90f96 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:13.028797 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b4f94-881d-4dd4-8d40-7081a0b90f96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:13.029101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b4f94-881d-4dd4-8d40-7081a0b90f96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:13.030504 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2b4f94-881d-4dd4-8d40-7081a0b90f96 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:13.043973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 543/1084] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:55:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:14.352225 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cebf8-43f7-4231-87ae-99220646055b None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:14.352778 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cebf8-43f7-4231-87ae-99220646055b None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:14.352778 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8cebf8-43f7-4231-87ae-99220646055b None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:14.354128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 542/1085] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:55:14.359548 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795ef50-960d-4757-8480-6e89c587670d None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:14.359958 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795ef50-960d-4757-8480-6e89c587670d None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:14.360179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b795ef50-960d-4757-8480-6e89c587670d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:14.378593 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-b795ef50-960d-4757-8480-6e89c587670d None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:14.379802 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-b795ef50-960d-4757-8480-6e89c587670d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:14.395092 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:14.451659 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 544/1086] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:55:14.474801 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8cebf8-43f7-4231-87ae-99220646055b None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:14.541492 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:14.586652 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8cebf8-43f7-4231-87ae-99220646055b None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:14.621531 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91781200-7c6e-4597-86bd-82bb95ae6b71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:14.622246 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91781200-7c6e-4597-86bd-82bb95ae6b71 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:14.622531 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91781200-7c6e-4597-86bd-82bb95ae6b71 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:14.622807 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91781200-7c6e-4597-86bd-82bb95ae6b71 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:14.633998 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91781200-7c6e-4597-86bd-82bb95ae6b71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:55:14.641635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-91781200-7c6e-4597-86bd-82bb95ae6b71 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:55:14.641635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:14.641635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:14.641635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:14.641635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:14.641635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:14.641635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:14.641635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:14.641635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:14.641635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:14.641635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:14.641635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:14.641635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:55:14.641635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:55:14.641635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:55:14.641635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:55:14.642673 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:55:14.642673 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:55:14.642673 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:55:14.642673 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:55:14.642673 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:14.642673 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:14.642673 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:14.642673 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:14.642673 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:14.642673 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:14.642673 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:14.642673 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:55:14.642673 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:14.642673 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:14.642673 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:14.642673 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:14.643572 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:14.643572 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:14.643572 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:14.643572 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:14.643572 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:55:14.643572 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:55:14.643572 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:55:14.643572 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:55:14.643572 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:55:14.643572 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:55:14.643572 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:55:14.643572 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:14.643572 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 543/1087] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:55:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:55:14.649158 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795ef50-960d-4757-8480-6e89c587670d None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:14.750592 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b795ef50-960d-4757-8480-6e89c587670d None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:14.786732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f40343-6420-44e6-81ed-4f32a0353b7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:14.787433 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f40343-6420-44e6-81ed-4f32a0353b7a None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:14.787714 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f40343-6420-44e6-81ed-4f32a0353b7a None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:14.787990 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f40343-6420-44e6-81ed-4f32a0353b7a None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:14.788751 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f40343-6420-44e6-81ed-4f32a0353b7a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:14.789059 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f40343-6420-44e6-81ed-4f32a0353b7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:14.790459 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f40343-6420-44e6-81ed-4f32a0353b7a None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:14.798856 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 545/1088] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:55:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:14.804883 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91781200-7c6e-4597-86bd-82bb95ae6b71 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:14.949186 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91781200-7c6e-4597-86bd-82bb95ae6b71 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:14.970941 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:14.985867 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aacf79a-2d80-4d3d-8d75-f6cb5334f9ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:14.986765 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aacf79a-2d80-4d3d-8d75-f6cb5334f9ee None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:14.987079 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aacf79a-2d80-4d3d-8d75-f6cb5334f9ee None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:14.987356 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aacf79a-2d80-4d3d-8d75-f6cb5334f9ee None admin] PATH_INFO: `/v3/users/1a76292e679a48799d615853a42c8e04` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:14.988551 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aacf79a-2d80-4d3d-8d75-f6cb5334f9ee None admin] RBAC: Authorizing `identity:get_user(user_id=1a76292e679a48799d615853a42c8e04)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:14.990003 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aacf79a-2d80-4d3d-8d75-f6cb5334f9ee None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:14.991837 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 544/1089] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:55:14 2022] GET /identity/v3/users/1a76292e679a48799d615853a42c8e04 => generated 285 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:14.997459 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f40343-6420-44e6-81ed-4f32a0353b7a None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:15.014159 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:15.055582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba1aea-59bc-4fb2-8c40-4d41c4135a24 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:15.055879 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba1aea-59bc-4fb2-8c40-4d41c4135a24 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:15.056148 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ba1aea-59bc-4fb2-8c40-4d41c4135a24 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:15.057700 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 545/1090] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:55:15.064021 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528913af-9129-4254-959f-d843e6ab2315 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:15.064760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528913af-9129-4254-959f-d843e6ab2315 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:15.065574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528913af-9129-4254-959f-d843e6ab2315 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:15.085758 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-528913af-9129-4254-959f-d843e6ab2315 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:15.087018 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-528913af-9129-4254-959f-d843e6ab2315 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:15.105563 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f40343-6420-44e6-81ed-4f32a0353b7a None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:15.147158 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474bb2e-f7e7-40a0-828a-f163663e1e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:15.147827 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474bb2e-f7e7-40a0-828a-f163663e1e9a None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:15.148110 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474bb2e-f7e7-40a0-828a-f163663e1e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:15.148384 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6474bb2e-f7e7-40a0-828a-f163663e1e9a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:15.163234 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 546/1091] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 16:55:15.186545 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528913af-9129-4254-959f-d843e6ab2315 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:15.216981 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6474bb2e-f7e7-40a0-828a-f163663e1e9a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:15.218368 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6474bb2e-f7e7-40a0-828a-f163663e1e9a None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:15.232460 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:15.287430 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528913af-9129-4254-959f-d843e6ab2315 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:15.304322 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:15.321033 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00118994-def0-4b9c-8b38-a00a8077e2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:15.321490 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 546/1092] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:55:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 16:55:15.321843 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00118994-def0-4b9c-8b38-a00a8077e2ff None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:15.322118 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00118994-def0-4b9c-8b38-a00a8077e2ff None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:15.322386 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00118994-def0-4b9c-8b38-a00a8077e2ff None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:15.327035 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474bb2e-f7e7-40a0-828a-f163663e1e9a None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:15.333222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00118994-def0-4b9c-8b38-a00a8077e2ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:55:15.341245 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-00118994-def0-4b9c-8b38-a00a8077e2ff None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 16:55:15.341245 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:15.341245 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:15.341245 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:15.341245 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:15.341245 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:15.341245 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:15.341245 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:15.341245 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:15.341245 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:15.341245 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:15.341245 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:15.341245 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:55:15.341245 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:55:15.341245 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:55:15.341245 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:55:15.342194 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:55:15.342194 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:55:15.342194 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:55:15.342194 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:55:15.342194 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:15.342194 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:15.342194 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:15.342194 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:15.342194 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:15.342194 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:15.342194 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:15.342194 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:55:15.342194 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:15.342194 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:15.342194 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:15.342194 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:15.343340 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:15.343340 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:15.343340 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:15.343340 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:15.343340 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:55:15.343340 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:55:15.343340 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:55:15.343340 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:55:15.343340 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:55:15.343340 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:55:15.343340 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 16:55:15.343340 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:15.343340 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 547/1093] 10.222.0.13 () {56 vars in 1120 bytes} [Mon May 23 16:55:15 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:55:15.348355 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00118994-def0-4b9c-8b38-a00a8077e2ff None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:15.433012 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6474bb2e-f7e7-40a0-828a-f163663e1e9a None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:15.445926 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:15.457733 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00118994-def0-4b9c-8b38-a00a8077e2ff None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:15.467354 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956002c7-c41a-4fd0-8065-b4cba7eb1eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:15.468452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956002c7-c41a-4fd0-8065-b4cba7eb1eac None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:15.468861 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956002c7-c41a-4fd0-8065-b4cba7eb1eac None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:15.469143 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956002c7-c41a-4fd0-8065-b4cba7eb1eac None admin] PATH_INFO: `/v3/projects/163424ef91fc4893889ba520ee61a62d` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:15.470724 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956002c7-c41a-4fd0-8065-b4cba7eb1eac None admin] RBAC: Authorizing `identity:get_project(project_id=163424ef91fc4893889ba520ee61a62d)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:15.472199 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956002c7-c41a-4fd0-8065-b4cba7eb1eac None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:15.474085 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 547/1094] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:55:15 2022] GET /identity/v3/projects/163424ef91fc4893889ba520ee61a62d => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:55:15.479890 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956002c7-c41a-4fd0-8065-b4cba7eb1eac None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:15.496640 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7b96e-55be-408b-8c08-3e05ef319386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:15.497289 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7b96e-55be-408b-8c08-3e05ef319386 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:15.497584 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7b96e-55be-408b-8c08-3e05ef319386 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:15.497857 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a7b96e-55be-408b-8c08-3e05ef319386 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:15.498597 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a7b96e-55be-408b-8c08-3e05ef319386 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:15.498903 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a7b96e-55be-408b-8c08-3e05ef319386 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:15.500330 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a7b96e-55be-408b-8c08-3e05ef319386 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:15.508879 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 548/1095] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 23 16:55:15 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:55:15.514472 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7b96e-55be-408b-8c08-3e05ef319386 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:15.556156 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:15.582400 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956002c7-c41a-4fd0-8065-b4cba7eb1eac None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:15.616380 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a7b96e-55be-408b-8c08-3e05ef319386 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:15.619365 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b061a-8462-4d20-8a1e-ce846f3bea99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:15.620029 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b061a-8462-4d20-8a1e-ce846f3bea99 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:15.620309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b061a-8462-4d20-8a1e-ce846f3bea99 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:15.620586 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3b061a-8462-4d20-8a1e-ce846f3bea99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:15.621409 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b061a-8462-4d20-8a1e-ce846f3bea99 None admin] RBAC: Adding query filter params (role.id=17bbe7cc1c3641509de5e9dca8534102, scope.project.id=163424ef91fc4893889ba520ee61a62d, user.id=1a76292e679a48799d615853a42c8e04) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:15.621718 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b061a-8462-4d20-8a1e-ce846f3bea99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:15.623169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3b061a-8462-4d20-8a1e-ce846f3bea99 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:15.629951 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 548/1096] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:55:15 2022] GET /identity/v3/role_assignments?user.id=1a76292e679a48799d615853a42c8e04&scope.project.id=163424ef91fc4893889ba520ee61a62d&role.id=17bbe7cc1c3641509de5e9dca8534102 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:55:15.650789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22893d-bc7a-4567-9469-ea3234489087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:15.651633 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22893d-bc7a-4567-9469-ea3234489087 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:15.651913 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22893d-bc7a-4567-9469-ea3234489087 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:15.652183 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb22893d-bc7a-4567-9469-ea3234489087 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:15.664245 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb22893d-bc7a-4567-9469-ea3234489087 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:15.665730 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb22893d-bc7a-4567-9469-ea3234489087 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:15.684347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-bb22893d-bc7a-4567-9469-ea3234489087 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:55:15.684347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:15.684347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:15.684347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:15.684347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:15.684347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:15.684347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:15.684347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:15.684347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:15.684347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:15.684347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:15.684347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:15.684347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:55:15.684347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:55:15.684347 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:55:15.685363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:55:15.685363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:15.685363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:15.685363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:15.685363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:15.685363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:15.685363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:15.685363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:15.685363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:55:15.685363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:15.685363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:15.685363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:15.685363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:15.685363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:15.685363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:15.685363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:15.685363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:15.686248 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:55:15.686248 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:55:15.686248 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:55:15.686248 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:15.686248 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 549/1097] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:55:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:55:15.692007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b061a-8462-4d20-8a1e-ce846f3bea99 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:15.748002 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:15.808830 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3b061a-8462-4d20-8a1e-ce846f3bea99 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:15.845057 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801b028-eaa1-4eb1-bdde-b6b0c254b236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:15.845760 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801b028-eaa1-4eb1-bdde-b6b0c254b236 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:15.845944 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801b028-eaa1-4eb1-bdde-b6b0c254b236 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:15.846214 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801b028-eaa1-4eb1-bdde-b6b0c254b236 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:15.846950 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1801b028-eaa1-4eb1-bdde-b6b0c254b236 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:15.847275 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1801b028-eaa1-4eb1-bdde-b6b0c254b236 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:15.848646 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1801b028-eaa1-4eb1-bdde-b6b0c254b236 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:15.860631 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 549/1098] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:15.866599 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22893d-bc7a-4567-9469-ea3234489087 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:15.960995 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:15.970795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb22893d-bc7a-4567-9469-ea3234489087 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:16.006166 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abcda0-60b7-4a9d-b1b8-8c81f2c6c1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:16.007228 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abcda0-60b7-4a9d-b1b8-8c81f2c6c1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:16.007505 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abcda0-60b7-4a9d-b1b8-8c81f2c6c1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:16.007777 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89abcda0-60b7-4a9d-b1b8-8c81f2c6c1f7 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:16.015726 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89abcda0-60b7-4a9d-b1b8-8c81f2c6c1f7 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:16.017380 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89abcda0-60b7-4a9d-b1b8-8c81f2c6c1f7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:16.031446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-89abcda0-60b7-4a9d-b1b8-8c81f2c6c1f7 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 16:55:16.031446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:16.031446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:16.031446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:16.031446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:16.031446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:16.031446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:16.031446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:16.031446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:16.031446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:16.031446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:16.031446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:16.031446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:55:16.031446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:55:16.031446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:55:16.031446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:55:16.032512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:16.032512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:16.032512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:55:16.032512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:55:16.032512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:55:16.032512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:55:16.032512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:16.032512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:16.032512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:16.032512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:16.032512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:16.032512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:55:16.032512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:16.032512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:16.032512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:16.032512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:16.032512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:16.032512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:16.033728 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:16.033728 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:16.033728 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:55:16.033728 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:55:16.033728 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:55:16.033728 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:55:16.033728 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:55:16.033728 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:55:16.033728 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 16:55:16.033728 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:16.033728 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 550/1099] 10.222.0.13 () {56 vars in 1093 bytes} [Mon May 23 16:55:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:55:16.038366 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801b028-eaa1-4eb1-bdde-b6b0c254b236 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:16.137069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801b028-eaa1-4eb1-bdde-b6b0c254b236 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:16.139468 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:16.172125 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a5446-637f-43ae-94bf-aafd4f75ec44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:16.172790 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a5446-637f-43ae-94bf-aafd4f75ec44 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:16.173307 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a5446-637f-43ae-94bf-aafd4f75ec44 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:16.173594 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6a5446-637f-43ae-94bf-aafd4f75ec44 None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:16.174421 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a5446-637f-43ae-94bf-aafd4f75ec44 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:16.174730 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a5446-637f-43ae-94bf-aafd4f75ec44 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:16.176187 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6a5446-637f-43ae-94bf-aafd4f75ec44 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:16.193741 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 550/1100] 10.222.0.13 () {56 vars in 1133 bytes} [Mon May 23 16:55:16 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:16.199425 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abcda0-60b7-4a9d-b1b8-8c81f2c6c1f7 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:16.303035 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89abcda0-60b7-4a9d-b1b8-8c81f2c6c1f7 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:16.337327 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e6e4b-2b8a-441f-b45e-06acf5d08eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:16.337971 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e6e4b-2b8a-441f-b45e-06acf5d08eaa None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:16.338247 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e6e4b-2b8a-441f-b45e-06acf5d08eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:16.338511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-807e6e4b-2b8a-441f-b45e-06acf5d08eaa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:16.359741 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:16.407647 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e6e4b-2b8a-441f-b45e-06acf5d08eaa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:16.409268 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-807e6e4b-2b8a-441f-b45e-06acf5d08eaa None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:16.516281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 551/1101] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:55:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 16:55:16.522288 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a5446-637f-43ae-94bf-aafd4f75ec44 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:16.623623 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6a5446-637f-43ae-94bf-aafd4f75ec44 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:16.659021 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8df36-26ec-43b2-b801-741c61f584cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:16.659806 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8df36-26ec-43b2-b801-741c61f584cf None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:16.660084 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8df36-26ec-43b2-b801-741c61f584cf None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:16.660358 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d8df36-26ec-43b2-b801-741c61f584cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:16.670585 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8df36-26ec-43b2-b801-741c61f584cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:16.672061 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1d8df36-26ec-43b2-b801-741c61f584cf None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:16.689644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-e1d8df36-26ec-43b2-b801-741c61f584cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:55:16.689644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:16.689644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:16.689644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:16.689644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:16.689644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:16.689644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:16.689644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:16.689644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:16.689644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:16.689644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:16.689644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:16.689644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:55:16.689644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:55:16.689644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:55:16.690749 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:55:16.690749 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:16.690749 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:16.690749 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:16.690749 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:16.690749 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:16.690749 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:16.690749 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:16.690749 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:55:16.690749 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:16.690749 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:16.690749 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:16.690749 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:16.690749 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:16.690749 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:16.690749 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:16.690749 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:16.696624 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:55:16.696624 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:55:16.696624 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:55:16.696624 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:55:16.696624 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 551/1102] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:55:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:55:16.698196 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e6e4b-2b8a-441f-b45e-06acf5d08eaa None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:16.755284 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:16.805569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-807e6e4b-2b8a-441f-b45e-06acf5d08eaa None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:16.841038 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89754bfa-2e89-41e0-921c-21afc63cd377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:16.841699 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89754bfa-2e89-41e0-921c-21afc63cd377 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:16.841985 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89754bfa-2e89-41e0-921c-21afc63cd377 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:16.842249 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89754bfa-2e89-41e0-921c-21afc63cd377 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:16.843002 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89754bfa-2e89-41e0-921c-21afc63cd377 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:16.843297 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89754bfa-2e89-41e0-921c-21afc63cd377 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:16.844708 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89754bfa-2e89-41e0-921c-21afc63cd377 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:16.856466 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 552/1103] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:55:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:16.862400 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8df36-26ec-43b2-b801-741c61f584cf None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:16.879513 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:16.973524 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d8df36-26ec-43b2-b801-741c61f584cf None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:17.009248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e6531-99ae-4eab-bb68-0d24dfa390b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:17.010032 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e6531-99ae-4eab-bb68-0d24dfa390b2 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:17.010309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e6531-99ae-4eab-bb68-0d24dfa390b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:17.010575 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e6531-99ae-4eab-bb68-0d24dfa390b2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:17.017285 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e6531-99ae-4eab-bb68-0d24dfa390b2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:17.018724 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e6531-99ae-4eab-bb68-0d24dfa390b2 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:17.032001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-101e6531-99ae-4eab-bb68-0d24dfa390b2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:55:17.032001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:17.032001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:17.032001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:17.032001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:17.032001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:17.032001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:17.032001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:17.032001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:17.032001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:17.032001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:17.032001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:17.032001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:55:17.032001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:55:17.032001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:55:17.032989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:55:17.032989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:17.032989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:17.032989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:17.032989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:17.032989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:17.032989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:17.032989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:17.032989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:55:17.032989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:17.032989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:17.032989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:17.032989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:17.032989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:17.032989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:17.032989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:17.032989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:17.033903 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:55:17.033903 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:55:17.033903 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:55:17.033903 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:55:17.033903 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:55:17.033903 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:55:17.033903 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:55:17.033903 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:55:17.033903 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 552/1104] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:55:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:55:17.039664 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89754bfa-2e89-41e0-921c-21afc63cd377 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:17.145168 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89754bfa-2e89-41e0-921c-21afc63cd377 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:17.158220 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:17.179816 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ca6c21-6bce-4af2-8b5f-7f5eabee890a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:17.180472 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca6c21-6bce-4af2-8b5f-7f5eabee890a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:17.180755 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca6c21-6bce-4af2-8b5f-7f5eabee890a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:17.181033 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ca6c21-6bce-4af2-8b5f-7f5eabee890a None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:17.181812 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ca6c21-6bce-4af2-8b5f-7f5eabee890a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:17.182130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ca6c21-6bce-4af2-8b5f-7f5eabee890a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:17.183616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ca6c21-6bce-4af2-8b5f-7f5eabee890a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:17.195565 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 553/1105] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:55:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:17.201623 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e6531-99ae-4eab-bb68-0d24dfa390b2 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:17.276808 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:17.302838 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e6531-99ae-4eab-bb68-0d24dfa390b2 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:17.338278 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c190d1f5-dbd1-4eb8-8857-1bb28ccb6478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:17.338938 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190d1f5-dbd1-4eb8-8857-1bb28ccb6478 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:17.339252 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190d1f5-dbd1-4eb8-8857-1bb28ccb6478 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:17.339527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c190d1f5-dbd1-4eb8-8857-1bb28ccb6478 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:17.340352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190d1f5-dbd1-4eb8-8857-1bb28ccb6478 None admin] RBAC: Adding query filter params (role.id=7141f022bbca499c80cc11cca858e064, scope.project.id=0595667fb2e04a3ca4716f669d6698b0, user.id=225c7b385f12445fab86b3b5606b1da1) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:17.340667 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190d1f5-dbd1-4eb8-8857-1bb28ccb6478 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:17.342095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c190d1f5-dbd1-4eb8-8857-1bb28ccb6478 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:17.348535 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 553/1106] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:55:17 2022] GET /identity/v3/role_assignments?user.id=225c7b385f12445fab86b3b5606b1da1&scope.project.id=0595667fb2e04a3ca4716f669d6698b0&role.id=7141f022bbca499c80cc11cca858e064 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:17.848349 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57521956-2d1a-4b96-a282-2cdb496fa3f5 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:17.848877 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57521956-2d1a-4b96-a282-2cdb496fa3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:17.848877 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57521956-2d1a-4b96-a282-2cdb496fa3f5 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:17.850286 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 554/1107] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:55:17.856153 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6eae5b-058a-45af-a7df-938ce6511b21 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:17.856606 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6eae5b-058a-45af-a7df-938ce6511b21 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:17.856703 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6eae5b-058a-45af-a7df-938ce6511b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:17.874892 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-7d6eae5b-058a-45af-a7df-938ce6511b21 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:17.876324 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-7d6eae5b-058a-45af-a7df-938ce6511b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:17.945097 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 554/1108] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:55:17.969886 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57521956-2d1a-4b96-a282-2cdb496fa3f5 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:18.086756 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57521956-2d1a-4b96-a282-2cdb496fa3f5 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:18.121511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77bc2440-04a8-4a99-8704-7279caa8b1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:18.122231 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc2440-04a8-4a99-8704-7279caa8b1af None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:18.122509 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc2440-04a8-4a99-8704-7279caa8b1af None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:18.122784 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77bc2440-04a8-4a99-8704-7279caa8b1af None admin] PATH_INFO: `/v3/domains/default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:18.123950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc2440-04a8-4a99-8704-7279caa8b1af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:18.125393 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77bc2440-04a8-4a99-8704-7279caa8b1af None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:18.127243 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 555/1109] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:55:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:18.133743 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6eae5b-058a-45af-a7df-938ce6511b21 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:18.200357 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:18.233483 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6eae5b-058a-45af-a7df-938ce6511b21 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:18.267498 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2feac99-6882-4e79-aa46-a8988486753a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:18.268191 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2feac99-6882-4e79-aa46-a8988486753a None admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:18.268465 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2feac99-6882-4e79-aa46-a8988486753a None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:18.268735 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2feac99-6882-4e79-aa46-a8988486753a None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:18.269438 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2feac99-6882-4e79-aa46-a8988486753a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:18.270861 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2feac99-6882-4e79-aa46-a8988486753a None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:18.295592 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 555/1110] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 23 16:55:18 2022] POST /identity/v3/users => generated 286 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:55:20.560777 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b01ce-22ab-4684-9d58-db5cad5fa6bf None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:20.561635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b01ce-22ab-4684-9d58-db5cad5fa6bf None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:20.561635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b01ce-22ab-4684-9d58-db5cad5fa6bf None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:20.564020 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 556/1111] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:55:20.572692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f8b38-f530-4838-9002-6e34fc774266 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:20.573413 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f8b38-f530-4838-9002-6e34fc774266 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:20.574184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978f8b38-f530-4838-9002-6e34fc774266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:20.599516 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-978f8b38-f530-4838-9002-6e34fc774266 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:20.604348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-978f8b38-f530-4838-9002-6e34fc774266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:20.678568 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 556/1112] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:55:20.712262 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b01ce-22ab-4684-9d58-db5cad5fa6bf None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:20.758057 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:20.832030 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b01ce-22ab-4684-9d58-db5cad5fa6bf None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:20.867831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035bb23-e046-4ce6-81e4-392789fd3c0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:20.868670 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035bb23-e046-4ce6-81e4-392789fd3c0c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:20.868944 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035bb23-e046-4ce6-81e4-392789fd3c0c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:20.869213 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6035bb23-e046-4ce6-81e4-392789fd3c0c None admin] PATH_INFO: `/v3/roles/6c30aca17fad4c4fa40b9461c5bda291` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:20.870784 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6035bb23-e046-4ce6-81e4-392789fd3c0c None admin] RBAC: Authorizing `identity:get_role(role_id=6c30aca17fad4c4fa40b9461c5bda291)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:20.872184 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6035bb23-e046-4ce6-81e4-392789fd3c0c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:20.873523 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 557/1113] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:55:20 2022] GET /identity/v3/roles/6c30aca17fad4c4fa40b9461c5bda291 => generated 223 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:20.879829 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f8b38-f530-4838-9002-6e34fc774266 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:20.936395 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:21.001876 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978f8b38-f530-4838-9002-6e34fc774266 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:21.038771 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a6377-00d3-4e27-9b6e-340d6b07baac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:21.039602 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a6377-00d3-4e27-9b6e-340d6b07baac None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:21.039882 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a6377-00d3-4e27-9b6e-340d6b07baac None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:21.040153 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-284a6377-00d3-4e27-9b6e-340d6b07baac None admin] PATH_INFO: `/v3/users/b43a50bbf4a34baca2e878127e322508` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:21.054017 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a6377-00d3-4e27-9b6e-340d6b07baac None admin] RBAC: Authorizing `identity:get_user(user_id=b43a50bbf4a34baca2e878127e322508)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:21.055501 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-284a6377-00d3-4e27-9b6e-340d6b07baac None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:21.057127 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 557/1114] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:55:20 2022] GET /identity/v3/users/b43a50bbf4a34baca2e878127e322508 => generated 286 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:55:21.065681 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035bb23-e046-4ce6-81e4-392789fd3c0c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:21.171453 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6035bb23-e046-4ce6-81e4-392789fd3c0c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:21.181103 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:21.210577 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc41ee-be16-43a3-93e5-e018ae581f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:21.211252 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc41ee-be16-43a3-93e5-e018ae581f72 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:21.211530 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc41ee-be16-43a3-93e5-e018ae581f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:21.211802 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89cc41ee-be16-43a3-93e5-e018ae581f72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:21.284610 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc41ee-be16-43a3-93e5-e018ae581f72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:21.286811 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89cc41ee-be16-43a3-93e5-e018ae581f72 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:21.405372 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 558/1115] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:55:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 16:55:21.415325 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a6377-00d3-4e27-9b6e-340d6b07baac None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:21.522218 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-284a6377-00d3-4e27-9b6e-340d6b07baac None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:21.536276 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:21.561079 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1dca9f-beff-4955-9b85-7e753cda55f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:21.561939 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dca9f-beff-4955-9b85-7e753cda55f2 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:21.562215 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dca9f-beff-4955-9b85-7e753cda55f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:21.562480 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1dca9f-beff-4955-9b85-7e753cda55f2 None admin] PATH_INFO: `/v3/projects/163424ef91fc4893889ba520ee61a62d` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:21.564213 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1dca9f-beff-4955-9b85-7e753cda55f2 None admin] RBAC: Authorizing `identity:get_project(project_id=163424ef91fc4893889ba520ee61a62d)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:21.565659 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1dca9f-beff-4955-9b85-7e753cda55f2 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:21.567381 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 558/1116] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:55:21 2022] GET /identity/v3/projects/163424ef91fc4893889ba520ee61a62d => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:21.573751 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc41ee-be16-43a3-93e5-e018ae581f72 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:21.584973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:21.681685 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89cc41ee-be16-43a3-93e5-e018ae581f72 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:21.717076 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548d96bb-d54f-4da8-8473-a68bb9c30e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:21.717714 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d96bb-d54f-4da8-8473-a68bb9c30e9e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:21.717996 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d96bb-d54f-4da8-8473-a68bb9c30e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:21.718264 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548d96bb-d54f-4da8-8473-a68bb9c30e9e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:21.719120 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d96bb-d54f-4da8-8473-a68bb9c30e9e None admin] RBAC: Adding query filter params (role.id=6c30aca17fad4c4fa40b9461c5bda291, scope.project.id=163424ef91fc4893889ba520ee61a62d, user.id=b43a50bbf4a34baca2e878127e322508) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:21.719432 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d96bb-d54f-4da8-8473-a68bb9c30e9e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:21.721016 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548d96bb-d54f-4da8-8473-a68bb9c30e9e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:21.726568 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 559/1117] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:55:21 2022] GET /identity/v3/role_assignments?user.id=b43a50bbf4a34baca2e878127e322508&scope.project.id=163424ef91fc4893889ba520ee61a62d&role.id=6c30aca17fad4c4fa40b9461c5bda291 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:55:23.867191 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f384154-de0a-4ec5-83df-f88fe20aaa44 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:23.868050 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f384154-de0a-4ec5-83df-f88fe20aaa44 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:23.868050 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f384154-de0a-4ec5-83df-f88fe20aaa44 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:23.870265 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 559/1118] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:55:23.876765 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f5a31-b599-41fb-bdaf-3ee6c0620332 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:23.877190 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f5a31-b599-41fb-bdaf-3ee6c0620332 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:23.877314 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6f5a31-b599-41fb-bdaf-3ee6c0620332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:23.897499 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-3f6f5a31-b599-41fb-bdaf-3ee6c0620332 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:23.898674 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-3f6f5a31-b599-41fb-bdaf-3ee6c0620332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:23.975237 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 560/1119] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:55:23.997989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f384154-de0a-4ec5-83df-f88fe20aaa44 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:24.109661 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f384154-de0a-4ec5-83df-f88fe20aaa44 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:24.146670 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453032d7-58f1-444d-a4aa-e1c346ba4bd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:24.147483 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453032d7-58f1-444d-a4aa-e1c346ba4bd1 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:24.147762 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453032d7-58f1-444d-a4aa-e1c346ba4bd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:24.148031 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453032d7-58f1-444d-a4aa-e1c346ba4bd1 None admin] PATH_INFO: `/v3/roles/6c30aca17fad4c4fa40b9461c5bda291` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:24.149601 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453032d7-58f1-444d-a4aa-e1c346ba4bd1 None admin] RBAC: Authorizing `identity:get_role(role_id=6c30aca17fad4c4fa40b9461c5bda291)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:24.150956 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453032d7-58f1-444d-a4aa-e1c346ba4bd1 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:24.152336 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 560/1120] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:55:23 2022] GET /identity/v3/roles/6c30aca17fad4c4fa40b9461c5bda291 => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:24.158597 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f5a31-b599-41fb-bdaf-3ee6c0620332 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:24.160783 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:24.268280 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6f5a31-b599-41fb-bdaf-3ee6c0620332 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:24.303537 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdad43f-a2ae-42d6-877a-101ff8b2fcea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:24.304398 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdad43f-a2ae-42d6-877a-101ff8b2fcea None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:24.304678 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdad43f-a2ae-42d6-877a-101ff8b2fcea None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:24.304950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdad43f-a2ae-42d6-877a-101ff8b2fcea None admin] PATH_INFO: `/v3/users/b43a50bbf4a34baca2e878127e322508` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:24.306134 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdad43f-a2ae-42d6-877a-101ff8b2fcea None admin] RBAC: Authorizing `identity:get_user(user_id=b43a50bbf4a34baca2e878127e322508)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:24.307619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdad43f-a2ae-42d6-877a-101ff8b2fcea None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:24.309235 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 561/1121] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:55:24 2022] GET /identity/v3/users/b43a50bbf4a34baca2e878127e322508 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:24.316027 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453032d7-58f1-444d-a4aa-e1c346ba4bd1 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:24.364555 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:24.420716 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453032d7-58f1-444d-a4aa-e1c346ba4bd1 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:24.457115 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34188875-98b1-4920-bee7-5c0400020743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:24.457964 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34188875-98b1-4920-bee7-5c0400020743 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:24.458244 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34188875-98b1-4920-bee7-5c0400020743 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:24.458519 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34188875-98b1-4920-bee7-5c0400020743 None admin] PATH_INFO: `/v3/projects/163424ef91fc4893889ba520ee61a62d` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:24.459767 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34188875-98b1-4920-bee7-5c0400020743 None admin] RBAC: Authorizing `identity:get_project(project_id=163424ef91fc4893889ba520ee61a62d)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:24.461194 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34188875-98b1-4920-bee7-5c0400020743 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:24.462885 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 561/1122] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:55:24 2022] GET /identity/v3/projects/163424ef91fc4893889ba520ee61a62d => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:24.469184 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdad43f-a2ae-42d6-877a-101ff8b2fcea None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:24.542262 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:24.573919 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdad43f-a2ae-42d6-877a-101ff8b2fcea None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:24.621582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7bad41-b4fd-4ceb-b541-22979cd9df6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:24.622995 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bad41-b4fd-4ceb-b541-22979cd9df6f None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:24.623512 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bad41-b4fd-4ceb-b541-22979cd9df6f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:24.623972 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7bad41-b4fd-4ceb-b541-22979cd9df6f None admin] PATH_INFO: `/v3/projects/163424ef91fc4893889ba520ee61a62d/users/b43a50bbf4a34baca2e878127e322508/roles/6c30aca17fad4c4fa40b9461c5bda291` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:24.627635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7bad41-b4fd-4ceb-b541-22979cd9df6f None admin] RBAC: Authorizing `identity:create_grant(project_id=163424ef91fc4893889ba520ee61a62d, user_id=b43a50bbf4a34baca2e878127e322508, role_id=6c30aca17fad4c4fa40b9461c5bda291)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:24.629110 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7bad41-b4fd-4ceb-b541-22979cd9df6f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:24.647360 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 562/1123] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:55:24 2022] PUT /identity/v3/projects/163424ef91fc4893889ba520ee61a62d/users/b43a50bbf4a34baca2e878127e322508/roles/6c30aca17fad4c4fa40b9461c5bda291 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:55:26.789007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b8e6c-a630-4eca-a2f0-c242ea080a1c None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:26.789546 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b8e6c-a630-4eca-a2f0-c242ea080a1c None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:26.789546 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-674b8e6c-a630-4eca-a2f0-c242ea080a1c None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:26.790805 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 562/1124] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:55:26.797306 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7582d-5050-45a9-803e-770d163a6015 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:26.798101 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7582d-5050-45a9-803e-770d163a6015 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:26.798291 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b7582d-5050-45a9-803e-770d163a6015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:26.821668 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-90b7582d-5050-45a9-803e-770d163a6015 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:26.822848 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-90b7582d-5050-45a9-803e-770d163a6015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:26.893594 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 563/1125] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:55:26.917693 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b8e6c-a630-4eca-a2f0-c242ea080a1c None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:27.023514 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-674b8e6c-a630-4eca-a2f0-c242ea080a1c None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:27.041359 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:27.063916 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc05fa-3bc2-4d01-aef2-799654969bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:27.064785 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc05fa-3bc2-4d01-aef2-799654969bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:27.065062 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc05fa-3bc2-4d01-aef2-799654969bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:27.065319 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc05fa-3bc2-4d01-aef2-799654969bb5 None admin] PATH_INFO: `/v3/roles/6c30aca17fad4c4fa40b9461c5bda291` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:27.066929 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc05fa-3bc2-4d01-aef2-799654969bb5 None admin] RBAC: Authorizing `identity:get_role(role_id=6c30aca17fad4c4fa40b9461c5bda291)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:27.068340 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc05fa-3bc2-4d01-aef2-799654969bb5 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:27.069709 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 563/1126] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:55:26 2022] GET /identity/v3/roles/6c30aca17fad4c4fa40b9461c5bda291 => generated 223 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:27.076522 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7582d-5050-45a9-803e-770d163a6015 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:27.143286 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:27.197644 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b7582d-5050-45a9-803e-770d163a6015 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:27.236318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5287d-4c7a-43e5-990a-b6018b1f536e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:27.237191 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5287d-4c7a-43e5-990a-b6018b1f536e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:27.237472 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5287d-4c7a-43e5-990a-b6018b1f536e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:27.237745 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e5287d-4c7a-43e5-990a-b6018b1f536e None admin] PATH_INFO: `/v3/users/b43a50bbf4a34baca2e878127e322508` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:27.239111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5287d-4c7a-43e5-990a-b6018b1f536e None admin] RBAC: Authorizing `identity:get_user(user_id=b43a50bbf4a34baca2e878127e322508)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:27.240561 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e5287d-4c7a-43e5-990a-b6018b1f536e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:27.242196 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 564/1127] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:55:27 2022] GET /identity/v3/users/b43a50bbf4a34baca2e878127e322508 => generated 286 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:27.251352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc05fa-3bc2-4d01-aef2-799654969bb5 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:27.358658 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc05fa-3bc2-4d01-aef2-799654969bb5 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:27.393633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b54ce-d47e-46a7-98a4-b44f5db5283c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:27.394270 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b54ce-d47e-46a7-98a4-b44f5db5283c None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:27.394553 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b54ce-d47e-46a7-98a4-b44f5db5283c None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:27.394828 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41b54ce-d47e-46a7-98a4-b44f5db5283c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:27.447539 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:27.465816 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b54ce-d47e-46a7-98a4-b44f5db5283c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:27.467298 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41b54ce-d47e-46a7-98a4-b44f5db5283c None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:27.571294 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 564/1128] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:55:27 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 16:55:27.577699 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5287d-4c7a-43e5-990a-b6018b1f536e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:27.685739 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e5287d-4c7a-43e5-990a-b6018b1f536e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:27.712681 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:27.724465 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdeee2f-930c-47d6-9830-c191e3a31900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:27.725329 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdeee2f-930c-47d6-9830-c191e3a31900 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:27.725606 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdeee2f-930c-47d6-9830-c191e3a31900 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:27.725872 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdeee2f-930c-47d6-9830-c191e3a31900 None admin] PATH_INFO: `/v3/projects/163424ef91fc4893889ba520ee61a62d` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:27.727205 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdeee2f-930c-47d6-9830-c191e3a31900 None admin] RBAC: Authorizing `identity:get_project(project_id=163424ef91fc4893889ba520ee61a62d)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:27.728650 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdeee2f-930c-47d6-9830-c191e3a31900 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:27.730320 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 565/1129] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:55:27 2022] GET /identity/v3/projects/163424ef91fc4893889ba520ee61a62d => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:27.736493 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b54ce-d47e-46a7-98a4-b44f5db5283c None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:27.825106 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:27.843516 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41b54ce-d47e-46a7-98a4-b44f5db5283c None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:27.882732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba54c14-96ff-4e9c-9d62-bf159443d000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:27.883401 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba54c14-96ff-4e9c-9d62-bf159443d000 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:27.883676 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba54c14-96ff-4e9c-9d62-bf159443d000 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:27.883945 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba54c14-96ff-4e9c-9d62-bf159443d000 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:27.884754 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba54c14-96ff-4e9c-9d62-bf159443d000 None admin] RBAC: Adding query filter params (role.id=6c30aca17fad4c4fa40b9461c5bda291, scope.project.id=163424ef91fc4893889ba520ee61a62d, user.id=b43a50bbf4a34baca2e878127e322508) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:27.885064 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba54c14-96ff-4e9c-9d62-bf159443d000 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:27.886485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba54c14-96ff-4e9c-9d62-bf159443d000 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:27.892662 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 565/1130] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:55:27 2022] GET /identity/v3/role_assignments?user.id=b43a50bbf4a34baca2e878127e322508&scope.project.id=163424ef91fc4893889ba520ee61a62d&role.id=6c30aca17fad4c4fa40b9461c5bda291 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:30.077572 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d4cde-b16c-4d7b-8e1f-b1062654e11f None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:30.078132 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d4cde-b16c-4d7b-8e1f-b1062654e11f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:30.078132 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1d4cde-b16c-4d7b-8e1f-b1062654e11f None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:30.079598 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 566/1131] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:55:30.085953 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75485e-3e66-4281-90f3-dea7016878b1 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:30.086378 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75485e-3e66-4281-90f3-dea7016878b1 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:30.086527 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b75485e-3e66-4281-90f3-dea7016878b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:30.112776 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-1b75485e-3e66-4281-90f3-dea7016878b1 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:30.114013 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-1b75485e-3e66-4281-90f3-dea7016878b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:30.188815 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 566/1132] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:55:30.218181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d4cde-b16c-4d7b-8e1f-b1062654e11f None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:30.321770 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1d4cde-b16c-4d7b-8e1f-b1062654e11f None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:30.356723 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56108fa-de8c-499b-8403-28af9dca81d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:30.357497 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56108fa-de8c-499b-8403-28af9dca81d4 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:30.357777 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56108fa-de8c-499b-8403-28af9dca81d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:30.358040 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56108fa-de8c-499b-8403-28af9dca81d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:30.359271 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56108fa-de8c-499b-8403-28af9dca81d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:30.360765 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56108fa-de8c-499b-8403-28af9dca81d4 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:30.362341 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 567/1133] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:55:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:30.368966 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75485e-3e66-4281-90f3-dea7016878b1 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:30.454448 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:30.478864 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b75485e-3e66-4281-90f3-dea7016878b1 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:30.514548 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261a7f8c-d084-40b9-9631-3a501a4a7b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:30.515238 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a7f8c-d084-40b9-9631-3a501a4a7b02 None admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:30.515517 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a7f8c-d084-40b9-9631-3a501a4a7b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:30.515786 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261a7f8c-d084-40b9-9631-3a501a4a7b02 None admin] PATH_INFO: `/v3/projects` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:30.516508 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a7f8c-d084-40b9-9631-3a501a4a7b02 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:30.517961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261a7f8c-d084-40b9-9631-3a501a4a7b02 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:30.548795 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 567/1134] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 23 16:55:30 2022] POST /identity/v3/projects => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:55:32.776492 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9c4dd-5b6d-4d20-b4f0-233b301b0aa6 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:32.777331 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9c4dd-5b6d-4d20-b4f0-233b301b0aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:32.777331 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9c4dd-5b6d-4d20-b4f0-233b301b0aa6 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:32.779475 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 568/1135] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:55:32.787061 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c660bca-28de-4f9b-a08f-f6d213de6ba9 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:32.787465 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c660bca-28de-4f9b-a08f-f6d213de6ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:32.787692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c660bca-28de-4f9b-a08f-f6d213de6ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:32.815653 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-2c660bca-28de-4f9b-a08f-f6d213de6ba9 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:32.816818 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-2c660bca-28de-4f9b-a08f-f6d213de6ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:32.898919 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 568/1136] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:55:32.926982 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9c4dd-5b6d-4d20-b4f0-233b301b0aa6 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:32.996437 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:33.051113 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9c4dd-5b6d-4d20-b4f0-233b301b0aa6 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:33.090745 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c142b97e-e10d-4e9f-8b92-e3583b671ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:33.091570 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142b97e-e10d-4e9f-8b92-e3583b671ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:33.091846 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142b97e-e10d-4e9f-8b92-e3583b671ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:33.092124 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c142b97e-e10d-4e9f-8b92-e3583b671ff2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:33.093520 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142b97e-e10d-4e9f-8b92-e3583b671ff2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:33.095048 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c142b97e-e10d-4e9f-8b92-e3583b671ff2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:33.096676 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 569/1137] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:55:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:33.103357 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c660bca-28de-4f9b-a08f-f6d213de6ba9 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:33.177104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:33.223737 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c660bca-28de-4f9b-a08f-f6d213de6ba9 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:33.259551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6918dccd-1e75-4ed4-8bee-c994976cb5ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:33.260244 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918dccd-1e75-4ed4-8bee-c994976cb5ef None admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:33.260522 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918dccd-1e75-4ed4-8bee-c994976cb5ef None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:33.260790 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6918dccd-1e75-4ed4-8bee-c994976cb5ef None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:33.261498 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918dccd-1e75-4ed4-8bee-c994976cb5ef None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:33.262955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6918dccd-1e75-4ed4-8bee-c994976cb5ef None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:33.284424 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 569/1138] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 23 16:55:33 2022] POST /identity/v3/users => generated 286 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:55:35.486812 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff39f4-867a-43d4-af20-fef30e689214 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:35.487350 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff39f4-867a-43d4-af20-fef30e689214 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:35.487350 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ff39f4-867a-43d4-af20-fef30e689214 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:35.488797 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 570/1139] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:55:35.494167 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a7176-b2a5-45e6-8519-af6966a10862 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:35.494602 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a7176-b2a5-45e6-8519-af6966a10862 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:35.494692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0a7176-b2a5-45e6-8519-af6966a10862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:35.512907 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-ab0a7176-b2a5-45e6-8519-af6966a10862 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:35.514058 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-ab0a7176-b2a5-45e6-8519-af6966a10862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:35.608521 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 570/1140] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:55:35.633090 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff39f4-867a-43d4-af20-fef30e689214 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:35.745932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ff39f4-867a-43d4-af20-fef30e689214 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:35.776382 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:35.783063 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68801fbf-2b3d-4d53-9df1-950caf7f7b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:35.783858 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68801fbf-2b3d-4d53-9df1-950caf7f7b5c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:35.784131 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68801fbf-2b3d-4d53-9df1-950caf7f7b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:35.784392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68801fbf-2b3d-4d53-9df1-950caf7f7b5c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:35.794766 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68801fbf-2b3d-4d53-9df1-950caf7f7b5c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:55:35.802490 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-68801fbf-2b3d-4d53-9df1-950caf7f7b5c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:55:35.802490 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:35.802490 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:35.802490 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:35.802490 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:35.802490 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:35.802490 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:35.802490 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:35.802490 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:35.802490 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:35.802490 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:35.802490 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:35.802490 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:55:35.802490 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:55:35.802490 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:55:35.802490 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:55:35.803670 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:55:35.803670 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:55:35.803670 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:55:35.803670 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:55:35.803670 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:35.803670 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:35.803670 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:35.803670 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:35.803670 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:35.803670 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:35.803670 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:35.803670 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:55:35.803670 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:35.803670 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:35.803670 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:35.803670 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:35.804705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:35.804705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:35.804705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:35.804705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:35.804705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:55:35.804705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:55:35.804705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:55:35.804705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:55:35.804705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:55:35.804705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:55:35.804705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:55:35.804705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:35.804705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 571/1141] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:55:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:55:35.809958 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a7176-b2a5-45e6-8519-af6966a10862 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:35.856794 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:35.912438 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0a7176-b2a5-45e6-8519-af6966a10862 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:35.947518 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1621b-8d8e-484a-8165-dff1438fa373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:35.948167 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1621b-8d8e-484a-8165-dff1438fa373 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:35.948443 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1621b-8d8e-484a-8165-dff1438fa373 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:35.948719 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf1621b-8d8e-484a-8165-dff1438fa373 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:35.949452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1621b-8d8e-484a-8165-dff1438fa373 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:35.949758 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1621b-8d8e-484a-8165-dff1438fa373 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:35.951153 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf1621b-8d8e-484a-8165-dff1438fa373 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:35.959589 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 571/1142] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:55:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:35.965979 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68801fbf-2b3d-4d53-9df1-950caf7f7b5c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:36.073096 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68801fbf-2b3d-4d53-9df1-950caf7f7b5c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:36.111926 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f2eda-7b03-4425-8aa1-26dd019306e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:36.112792 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f2eda-7b03-4425-8aa1-26dd019306e7 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:36.113065 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f2eda-7b03-4425-8aa1-26dd019306e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:36.113333 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901f2eda-7b03-4425-8aa1-26dd019306e7 None admin] PATH_INFO: `/v3/users/429176f1a9f549a88d4aaf5bfd64723d` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:36.130028 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f2eda-7b03-4425-8aa1-26dd019306e7 None admin] RBAC: Authorizing `identity:get_user(user_id=429176f1a9f549a88d4aaf5bfd64723d)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:36.131837 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901f2eda-7b03-4425-8aa1-26dd019306e7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:36.133465 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 572/1143] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:55:35 2022] GET /identity/v3/users/429176f1a9f549a88d4aaf5bfd64723d => generated 286 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:36.139302 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1621b-8d8e-484a-8165-dff1438fa373 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:36.243435 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf1621b-8d8e-484a-8165-dff1438fa373 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:36.252380 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:36.278886 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01938d68-c5f6-4ee4-b80b-7836f5e0cb2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:36.279558 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01938d68-c5f6-4ee4-b80b-7836f5e0cb2e None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:36.279855 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01938d68-c5f6-4ee4-b80b-7836f5e0cb2e None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:36.280114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01938d68-c5f6-4ee4-b80b-7836f5e0cb2e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:36.351567 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01938d68-c5f6-4ee4-b80b-7836f5e0cb2e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:36.353004 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01938d68-c5f6-4ee4-b80b-7836f5e0cb2e None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:36.455746 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 572/1144] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:55:36 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 16:55:36.461908 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f2eda-7b03-4425-8aa1-26dd019306e7 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:36.523070 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:36.575823 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901f2eda-7b03-4425-8aa1-26dd019306e7 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:36.616574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff22f51c-86c9-4de1-a1ae-c1a3c95b22ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:36.617381 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22f51c-86c9-4de1-a1ae-c1a3c95b22ad None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:36.617658 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22f51c-86c9-4de1-a1ae-c1a3c95b22ad None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:36.617931 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff22f51c-86c9-4de1-a1ae-c1a3c95b22ad None admin] PATH_INFO: `/v3/projects/533191f59405403c99fa6656aab8e9fe` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:36.619270 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff22f51c-86c9-4de1-a1ae-c1a3c95b22ad None admin] RBAC: Authorizing `identity:get_project(project_id=533191f59405403c99fa6656aab8e9fe)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:36.620704 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff22f51c-86c9-4de1-a1ae-c1a3c95b22ad None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:36.622389 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 573/1145] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:55:36 2022] GET /identity/v3/projects/533191f59405403c99fa6656aab8e9fe => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:55:36.629826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01938d68-c5f6-4ee4-b80b-7836f5e0cb2e None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:36.641294 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:36.734435 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01938d68-c5f6-4ee4-b80b-7836f5e0cb2e None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:36.771878 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355bbf7a-9f13-4e09-ac31-1bb26b4b57d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:36.772510 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bbf7a-9f13-4e09-ac31-1bb26b4b57d9 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:36.772781 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bbf7a-9f13-4e09-ac31-1bb26b4b57d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:36.773054 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355bbf7a-9f13-4e09-ac31-1bb26b4b57d9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:36.773870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355bbf7a-9f13-4e09-ac31-1bb26b4b57d9 None admin] RBAC: Adding query filter params (role.id=17bbe7cc1c3641509de5e9dca8534102, scope.project.id=533191f59405403c99fa6656aab8e9fe, user.id=429176f1a9f549a88d4aaf5bfd64723d) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:36.774179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355bbf7a-9f13-4e09-ac31-1bb26b4b57d9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:36.775633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355bbf7a-9f13-4e09-ac31-1bb26b4b57d9 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:36.781102 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 573/1146] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:55:36 2022] GET /identity/v3/role_assignments?user.id=429176f1a9f549a88d4aaf5bfd64723d&scope.project.id=533191f59405403c99fa6656aab8e9fe&role.id=17bbe7cc1c3641509de5e9dca8534102 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:39.019084 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303e3e8-3eb3-4fcc-9776-0e50c47f1791 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:39.019930 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303e3e8-3eb3-4fcc-9776-0e50c47f1791 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:39.019930 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d303e3e8-3eb3-4fcc-9776-0e50c47f1791 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:39.022126 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 574/1147] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:55:39.028020 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7886d1-2c53-4a93-9f7a-07150a5c0290 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:39.028462 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7886d1-2c53-4a93-9f7a-07150a5c0290 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:39.028974 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e7886d1-2c53-4a93-9f7a-07150a5c0290 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:39.057192 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-0e7886d1-2c53-4a93-9f7a-07150a5c0290 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:39.060365 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-0e7886d1-2c53-4a93-9f7a-07150a5c0290 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:39.145877 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 574/1148] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:55:39.170564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303e3e8-3eb3-4fcc-9776-0e50c47f1791 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:39.296291 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d303e3e8-3eb3-4fcc-9776-0e50c47f1791 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:39.317763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:39.338515 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09072375-d2ca-4d16-af04-9ad948562b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:39.339453 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09072375-d2ca-4d16-af04-9ad948562b9c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:39.339736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09072375-d2ca-4d16-af04-9ad948562b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:39.340008 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09072375-d2ca-4d16-af04-9ad948562b9c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:39.352050 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09072375-d2ca-4d16-af04-9ad948562b9c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:55:39.364084 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-09072375-d2ca-4d16-af04-9ad948562b9c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:55:39.364084 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:39.364084 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:39.364084 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:39.364084 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:39.364084 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:39.364084 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:39.364084 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:39.364084 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:39.364084 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:39.364084 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:39.364084 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:39.364084 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:55:39.364084 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:55:39.364084 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:55:39.364084 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:55:39.365221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:55:39.365221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:55:39.365221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:55:39.365221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:55:39.365221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:39.365221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:39.365221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:39.365221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:39.365221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:39.365221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:39.365221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:39.365221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:55:39.365221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:39.365221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:39.365221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:39.365221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:39.367187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:39.367187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:39.367187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:39.367187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:39.367187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:55:39.367187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:55:39.367187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:55:39.367187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:55:39.367187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:55:39.367187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:55:39.367187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:55:39.367187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:39.367187 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 575/1149] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:55:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:55:39.372920 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7886d1-2c53-4a93-9f7a-07150a5c0290 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:39.376203 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:39.494344 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e7886d1-2c53-4a93-9f7a-07150a5c0290 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:39.538431 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a2ae3-cadc-4943-8b76-1e450fd3e391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:39.539135 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a2ae3-cadc-4943-8b76-1e450fd3e391 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:39.539421 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a2ae3-cadc-4943-8b76-1e450fd3e391 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:39.539654 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a2ae3-cadc-4943-8b76-1e450fd3e391 None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:39.540440 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a2ae3-cadc-4943-8b76-1e450fd3e391 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:39.540752 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a2ae3-cadc-4943-8b76-1e450fd3e391 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:39.542145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a2ae3-cadc-4943-8b76-1e450fd3e391 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:39.551658 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 575/1150] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:55:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:39.560181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09072375-d2ca-4d16-af04-9ad948562b9c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:39.678747 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09072375-d2ca-4d16-af04-9ad948562b9c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:39.714230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172fe07-2a4c-4ca2-9040-bd7c2c736d02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:39.715132 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172fe07-2a4c-4ca2-9040-bd7c2c736d02 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:39.715417 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172fe07-2a4c-4ca2-9040-bd7c2c736d02 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:39.715691 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2172fe07-2a4c-4ca2-9040-bd7c2c736d02 None admin] PATH_INFO: `/v3/users/429176f1a9f549a88d4aaf5bfd64723d` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:39.716898 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2172fe07-2a4c-4ca2-9040-bd7c2c736d02 None admin] RBAC: Authorizing `identity:get_user(user_id=429176f1a9f549a88d4aaf5bfd64723d)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:39.718353 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2172fe07-2a4c-4ca2-9040-bd7c2c736d02 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:39.720140 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 576/1151] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:55:39 2022] GET /identity/v3/users/429176f1a9f549a88d4aaf5bfd64723d => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:39.726274 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a2ae3-cadc-4943-8b76-1e450fd3e391 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:39.794480 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:39.842312 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a2ae3-cadc-4943-8b76-1e450fd3e391 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:39.879976 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa16701-f5e5-43c8-b913-633f7c50bf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:39.880833 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa16701-f5e5-43c8-b913-633f7c50bf22 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:39.881112 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa16701-f5e5-43c8-b913-633f7c50bf22 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:39.881694 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa16701-f5e5-43c8-b913-633f7c50bf22 None admin] PATH_INFO: `/v3/projects/533191f59405403c99fa6656aab8e9fe` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:39.882988 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa16701-f5e5-43c8-b913-633f7c50bf22 None admin] RBAC: Authorizing `identity:get_project(project_id=533191f59405403c99fa6656aab8e9fe)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:39.884455 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa16701-f5e5-43c8-b913-633f7c50bf22 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:39.886140 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 576/1152] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:55:39 2022] GET /identity/v3/projects/533191f59405403c99fa6656aab8e9fe => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:39.892605 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172fe07-2a4c-4ca2-9040-bd7c2c736d02 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:39.941885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:40.005683 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2172fe07-2a4c-4ca2-9040-bd7c2c736d02 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:40.043921 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867f0b90-7bff-43be-9c41-ebad7652ac06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:40.044813 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f0b90-7bff-43be-9c41-ebad7652ac06 None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:40.045067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f0b90-7bff-43be-9c41-ebad7652ac06 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:40.045344 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867f0b90-7bff-43be-9c41-ebad7652ac06 None admin] PATH_INFO: `/v3/projects/533191f59405403c99fa6656aab8e9fe/users/429176f1a9f549a88d4aaf5bfd64723d/roles/17bbe7cc1c3641509de5e9dca8534102` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:40.047816 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867f0b90-7bff-43be-9c41-ebad7652ac06 None admin] RBAC: Authorizing `identity:create_grant(project_id=533191f59405403c99fa6656aab8e9fe, user_id=429176f1a9f549a88d4aaf5bfd64723d, role_id=17bbe7cc1c3641509de5e9dca8534102)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:40.049264 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867f0b90-7bff-43be-9c41-ebad7652ac06 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:40.065202 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 577/1153] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:55:39 2022] PUT /identity/v3/projects/533191f59405403c99fa6656aab8e9fe/users/429176f1a9f549a88d4aaf5bfd64723d/roles/17bbe7cc1c3641509de5e9dca8534102 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:55:42.254544 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7add21-0af5-4442-85b6-267616a57f5b None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:42.255483 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7add21-0af5-4442-85b6-267616a57f5b None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:42.255483 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7add21-0af5-4442-85b6-267616a57f5b None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:42.258038 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 577/1154] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:55:42.264430 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687c967-53c9-46cb-8e15-2e13407f0590 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:42.265135 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687c967-53c9-46cb-8e15-2e13407f0590 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:42.265442 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687c967-53c9-46cb-8e15-2e13407f0590 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:42.283772 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-e687c967-53c9-46cb-8e15-2e13407f0590 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:42.284948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e687c967-53c9-46cb-8e15-2e13407f0590 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:42.357765 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 578/1155] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:55:42.382993 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7add21-0af5-4442-85b6-267616a57f5b None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:42.517659 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7add21-0af5-4442-85b6-267616a57f5b None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:42.548021 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:42.554008 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3c97d-d4ab-416b-b8f3-e9c7f17f12bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:42.554800 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3c97d-d4ab-416b-b8f3-e9c7f17f12bc None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:42.555110 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3c97d-d4ab-416b-b8f3-e9c7f17f12bc None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:42.555390 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a3c97d-d4ab-416b-b8f3-e9c7f17f12bc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:42.566196 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a3c97d-d4ab-416b-b8f3-e9c7f17f12bc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:55:42.573842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-f7a3c97d-d4ab-416b-b8f3-e9c7f17f12bc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:55:42.573842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:42.573842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:42.573842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:42.573842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:42.573842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:42.573842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:42.573842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:42.573842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:42.573842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:42.573842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:42.573842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:42.573842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:55:42.573842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:55:42.573842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:55:42.573842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:55:42.574886 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:55:42.574886 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:55:42.574886 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:55:42.574886 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:55:42.574886 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:42.574886 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:42.574886 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:42.574886 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:42.574886 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:42.574886 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:42.574886 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:42.574886 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:55:42.574886 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:42.574886 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:42.574886 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:42.574886 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:42.575806 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:42.575806 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:42.575806 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:42.575806 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:42.575806 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:55:42.575806 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:55:42.575806 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:55:42.575806 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:55:42.575806 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:55:42.575806 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:55:42.575806 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:55:42.575806 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:55:42.575806 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 578/1156] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:55:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:55:42.584029 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687c967-53c9-46cb-8e15-2e13407f0590 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:42.624158 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:42.697049 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687c967-53c9-46cb-8e15-2e13407f0590 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:42.735091 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4297b63-11c4-41ec-b89a-ee3f5c4bb0d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:42.735752 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4297b63-11c4-41ec-b89a-ee3f5c4bb0d3 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:42.736004 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4297b63-11c4-41ec-b89a-ee3f5c4bb0d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:42.736299 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4297b63-11c4-41ec-b89a-ee3f5c4bb0d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:42.737043 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4297b63-11c4-41ec-b89a-ee3f5c4bb0d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:42.737356 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4297b63-11c4-41ec-b89a-ee3f5c4bb0d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:42.738741 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4297b63-11c4-41ec-b89a-ee3f5c4bb0d3 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:42.747100 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 579/1157] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:55:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:42.754192 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3c97d-d4ab-416b-b8f3-e9c7f17f12bc None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:42.870088 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a3c97d-d4ab-416b-b8f3-e9c7f17f12bc None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:42.907160 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34891df9-c780-4044-842f-50c58513cbaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:42.908022 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34891df9-c780-4044-842f-50c58513cbaa None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:42.908303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34891df9-c780-4044-842f-50c58513cbaa None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:42.908576 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34891df9-c780-4044-842f-50c58513cbaa None admin] PATH_INFO: `/v3/users/429176f1a9f549a88d4aaf5bfd64723d` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:42.909766 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34891df9-c780-4044-842f-50c58513cbaa None admin] RBAC: Authorizing `identity:get_user(user_id=429176f1a9f549a88d4aaf5bfd64723d)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:42.911221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34891df9-c780-4044-842f-50c58513cbaa None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:42.912819 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 579/1158] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:55:42 2022] GET /identity/v3/users/429176f1a9f549a88d4aaf5bfd64723d => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:42.919455 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4297b63-11c4-41ec-b89a-ee3f5c4bb0d3 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:43.027358 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:43.033619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4297b63-11c4-41ec-b89a-ee3f5c4bb0d3 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:43.074241 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c0390c-232f-427c-9027-dd9b3787bed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:43.074876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0390c-232f-427c-9027-dd9b3787bed7 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:43.075194 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0390c-232f-427c-9027-dd9b3787bed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:43.075468 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c0390c-232f-427c-9027-dd9b3787bed7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:43.156662 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c0390c-232f-427c-9027-dd9b3787bed7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:43.158079 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c0390c-232f-427c-9027-dd9b3787bed7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:43.254823 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:43.260211 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 580/1159] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:55:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 16:55:43.266872 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34891df9-c780-4044-842f-50c58513cbaa None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:43.328697 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:43.371549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34891df9-c780-4044-842f-50c58513cbaa None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:43.410554 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deba7fe6-6dd8-4026-916e-5259024833db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:43.411373 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba7fe6-6dd8-4026-916e-5259024833db None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:43.411646 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba7fe6-6dd8-4026-916e-5259024833db None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:43.411916 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deba7fe6-6dd8-4026-916e-5259024833db None admin] PATH_INFO: `/v3/projects/533191f59405403c99fa6656aab8e9fe` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:43.413100 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba7fe6-6dd8-4026-916e-5259024833db None admin] RBAC: Authorizing `identity:get_project(project_id=533191f59405403c99fa6656aab8e9fe)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:43.414523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deba7fe6-6dd8-4026-916e-5259024833db None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:43.416250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 580/1160] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:55:43 2022] GET /identity/v3/projects/533191f59405403c99fa6656aab8e9fe => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:43.422995 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c0390c-232f-427c-9027-dd9b3787bed7 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:43.541904 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c0390c-232f-427c-9027-dd9b3787bed7 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:43.577237 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9406ad37-6230-41ef-b168-ff5f77b27ad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:43.577932 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406ad37-6230-41ef-b168-ff5f77b27ad5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:43.578188 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406ad37-6230-41ef-b168-ff5f77b27ad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:43.578462 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406ad37-6230-41ef-b168-ff5f77b27ad5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:43.579377 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9406ad37-6230-41ef-b168-ff5f77b27ad5 None admin] RBAC: Adding query filter params (role.id=17bbe7cc1c3641509de5e9dca8534102, scope.project.id=533191f59405403c99fa6656aab8e9fe, user.id=429176f1a9f549a88d4aaf5bfd64723d) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:43.579692 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9406ad37-6230-41ef-b168-ff5f77b27ad5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:43.581143 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9406ad37-6230-41ef-b168-ff5f77b27ad5 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:43.587101 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 581/1161] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:55:43 2022] GET /identity/v3/role_assignments?user.id=429176f1a9f549a88d4aaf5bfd64723d&scope.project.id=533191f59405403c99fa6656aab8e9fe&role.id=17bbe7cc1c3641509de5e9dca8534102 => generated 608 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:45.819958 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38f0db-d645-40c4-bc4a-9eda89f1bc93 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:45.820777 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38f0db-d645-40c4-bc4a-9eda89f1bc93 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:45.820777 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38f0db-d645-40c4-bc4a-9eda89f1bc93 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:45.822858 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 581/1162] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:55:45.830459 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04b839-4630-493c-9f31-8c30d34fff5c None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:45.830898 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04b839-4630-493c-9f31-8c30d34fff5c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:45.831016 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b04b839-4630-493c-9f31-8c30d34fff5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:45.849573 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-0b04b839-4630-493c-9f31-8c30d34fff5c None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:45.850716 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0b04b839-4630-493c-9f31-8c30d34fff5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:45.908325 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:45.919244 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 582/1163] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 16:55:45.942481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f38f0db-d645-40c4-bc4a-9eda89f1bc93 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:46.053168 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f38f0db-d645-40c4-bc4a-9eda89f1bc93 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:46.081210 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:46.090632 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26783bef-c3d0-4bae-98fb-7126929c28ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:46.091324 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26783bef-c3d0-4bae-98fb-7126929c28ac None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:46.091605 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26783bef-c3d0-4bae-98fb-7126929c28ac None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:46.091874 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26783bef-c3d0-4bae-98fb-7126929c28ac None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:46.163085 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26783bef-c3d0-4bae-98fb-7126929c28ac None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:46.164551 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26783bef-c3d0-4bae-98fb-7126929c28ac None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:46.268905 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 582/1164] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:55:45 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 16:55:46.276034 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b04b839-4630-493c-9f31-8c30d34fff5c None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:46.383841 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b04b839-4630-493c-9f31-8c30d34fff5c None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:46.418467 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744133aa-c402-4230-b1f2-a2865bb9e25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:46.419289 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744133aa-c402-4230-b1f2-a2865bb9e25e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:46.419562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744133aa-c402-4230-b1f2-a2865bb9e25e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:46.419805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744133aa-c402-4230-b1f2-a2865bb9e25e None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:46.427199 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744133aa-c402-4230-b1f2-a2865bb9e25e None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:46.428663 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744133aa-c402-4230-b1f2-a2865bb9e25e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:46.442934 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-744133aa-c402-4230-b1f2-a2865bb9e25e None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 23 16:55:46.442934 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:46.442934 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 23 16:55:46.442934 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 23 16:55:46.442934 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:55:46.442934 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:55:46.442934 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:55:46.442934 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:55:46.442934 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 23 16:55:46.442934 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:46.442934 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:55:46.442934 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:46.442934 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:46.442934 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:46.442934 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:46.442934 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:46.442934 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:46.444061 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:46.444061 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:46.444061 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:46.444061 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:46.444061 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:46.444061 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:46.444061 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:55:46.444061 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:55:46.444061 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:55:46.444061 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:55:46.444061 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:46.444061 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:46.444061 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:46.444061 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:46.444061 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:46.444061 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:46.445067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:46.445067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:55:46.445067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:46.445067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:46.445067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:46.445067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:46.445067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:46.445067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:46.445067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:46.445067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:46.445067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 23 16:55:46.445067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:55:46.445067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 23 16:55:46.445067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:55:46.445067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 583/1165] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 23 16:55:46 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:55:46.450732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26783bef-c3d0-4bae-98fb-7126929c28ac None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:46.484617 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:46.564601 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26783bef-c3d0-4bae-98fb-7126929c28ac None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:46.600677 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5223c30e-137a-434e-b34f-749dcd832e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:46.601324 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223c30e-137a-434e-b34f-749dcd832e60 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:46.601600 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223c30e-137a-434e-b34f-749dcd832e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:46.601876 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5223c30e-137a-434e-b34f-749dcd832e60 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:46.602579 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5223c30e-137a-434e-b34f-749dcd832e60 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:46.602904 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5223c30e-137a-434e-b34f-749dcd832e60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:46.604292 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5223c30e-137a-434e-b34f-749dcd832e60 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:46.611782 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 583/1166] 10.222.0.13 () {56 vars in 1115 bytes} [Mon May 23 16:55:46 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:48.776930 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c800d8-6c2f-449a-9024-dbd26d570aa1 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:48.777802 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c800d8-6c2f-449a-9024-dbd26d570aa1 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:48.777802 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c800d8-6c2f-449a-9024-dbd26d570aa1 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:48.779904 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 584/1167] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:55:48.786247 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bdade-403c-453e-b050-f584eb7dc155 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:48.786710 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bdade-403c-453e-b050-f584eb7dc155 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:48.786815 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317bdade-403c-453e-b050-f584eb7dc155 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:48.804244 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-317bdade-403c-453e-b050-f584eb7dc155 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:48.805420 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-317bdade-403c-453e-b050-f584eb7dc155 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:48.880075 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 584/1168] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 16:55:48.902900 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c800d8-6c2f-449a-9024-dbd26d570aa1 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:48.963340 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:49.015022 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c800d8-6c2f-449a-9024-dbd26d570aa1 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:49.054477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa90b3b-3a9b-4ada-a395-2f5122cd6ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:49.055091 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa90b3b-3a9b-4ada-a395-2f5122cd6ca3 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:49.055367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa90b3b-3a9b-4ada-a395-2f5122cd6ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:49.055637 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa90b3b-3a9b-4ada-a395-2f5122cd6ca3 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:49.056370 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa90b3b-3a9b-4ada-a395-2f5122cd6ca3 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:49.057750 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa90b3b-3a9b-4ada-a395-2f5122cd6ca3 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:49.080666 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 585/1169] 10.222.0.13 () {60 vars in 1138 bytes} [Mon May 23 16:55:48 2022] POST /identity/v3/domains => generated 267 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:55:51.268808 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc414ed-d3a5-4198-a1c4-d9056883ace0 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:51.269383 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc414ed-d3a5-4198-a1c4-d9056883ace0 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:51.269383 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc414ed-d3a5-4198-a1c4-d9056883ace0 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:51.270723 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 585/1170] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:55:51.277114 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c181ab0-023f-49f2-b90e-bc7112c5d131 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:51.277856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c181ab0-023f-49f2-b90e-bc7112c5d131 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:51.278001 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c181ab0-023f-49f2-b90e-bc7112c5d131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:51.303370 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-4c181ab0-023f-49f2-b90e-bc7112c5d131 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:51.304542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-4c181ab0-023f-49f2-b90e-bc7112c5d131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:51.379776 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 586/1171] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:55:51.407675 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc414ed-d3a5-4198-a1c4-d9056883ace0 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:51.432477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:51.523988 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc414ed-d3a5-4198-a1c4-d9056883ace0 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:51.560657 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb80437-38a0-4069-ab8a-2cbc4664e7d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:51.561507 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb80437-38a0-4069-ab8a-2cbc4664e7d3 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:51.561964 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb80437-38a0-4069-ab8a-2cbc4664e7d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:51.562108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb80437-38a0-4069-ab8a-2cbc4664e7d3 None admin] PATH_INFO: `/v3/domains/ee931ecec7974b96a332effaf929e30c` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:51.572967 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb80437-38a0-4069-ab8a-2cbc4664e7d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ee931ecec7974b96a332effaf929e30c)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:51.574423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb80437-38a0-4069-ab8a-2cbc4664e7d3 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:51.576063 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 586/1172] 10.222.0.13 () {56 vars in 1183 bytes} [Mon May 23 16:55:51 2022] GET /identity/v3/domains/ee931ecec7974b96a332effaf929e30c => generated 267 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:51.582644 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c181ab0-023f-49f2-b90e-bc7112c5d131 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:51.629119 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:51.689338 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c181ab0-023f-49f2-b90e-bc7112c5d131 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:51.723774 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b14615-ada7-4fe1-a34c-ac11ef7fde98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:51.724432 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b14615-ada7-4fe1-a34c-ac11ef7fde98 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:51.724678 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b14615-ada7-4fe1-a34c-ac11ef7fde98 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:51.724965 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b14615-ada7-4fe1-a34c-ac11ef7fde98 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:51.725642 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b14615-ada7-4fe1-a34c-ac11ef7fde98 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:51.727133 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b14615-ada7-4fe1-a34c-ac11ef7fde98 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:51.754858 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 587/1173] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 23 16:55:51 2022] POST /identity/v3/projects => generated 361 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:55:53.984069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21eca58-5cfc-4583-baed-f96e359cfdf8 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:53.984654 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21eca58-5cfc-4583-baed-f96e359cfdf8 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:53.984654 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21eca58-5cfc-4583-baed-f96e359cfdf8 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:53.985964 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 587/1174] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:55:53.992560 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40686e17-c3dd-4862-bb9d-01144405da81 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:53.993270 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40686e17-c3dd-4862-bb9d-01144405da81 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:53.993432 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40686e17-c3dd-4862-bb9d-01144405da81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:54.016019 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-40686e17-c3dd-4862-bb9d-01144405da81 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:54.017211 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-40686e17-c3dd-4862-bb9d-01144405da81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:54.090335 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 588/1175] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:55:54.122647 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21eca58-5cfc-4583-baed-f96e359cfdf8 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:54.214879 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:54.238333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21eca58-5cfc-4583-baed-f96e359cfdf8 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:54.281431 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7682d30b-601c-4d65-bd75-30d796463cb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:54.282164 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682d30b-601c-4d65-bd75-30d796463cb4 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:54.282442 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682d30b-601c-4d65-bd75-30d796463cb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:54.282712 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7682d30b-601c-4d65-bd75-30d796463cb4 None admin] PATH_INFO: `/v3/domains/ee931ecec7974b96a332effaf929e30c` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:54.284045 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682d30b-601c-4d65-bd75-30d796463cb4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=ee931ecec7974b96a332effaf929e30c)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:54.285480 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7682d30b-601c-4d65-bd75-30d796463cb4 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:54.287085 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 588/1176] 10.222.0.13 () {56 vars in 1183 bytes} [Mon May 23 16:55:54 2022] GET /identity/v3/domains/ee931ecec7974b96a332effaf929e30c => generated 267 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:54.293873 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40686e17-c3dd-4862-bb9d-01144405da81 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:54.320980 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:54.399730 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40686e17-c3dd-4862-bb9d-01144405da81 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:54.435491 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41f248e-f176-4f42-8926-1e54ff520d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:54.436159 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f248e-f176-4f42-8926-1e54ff520d43 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:54.436437 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f248e-f176-4f42-8926-1e54ff520d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:54.436747 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41f248e-f176-4f42-8926-1e54ff520d43 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:54.437485 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f248e-f176-4f42-8926-1e54ff520d43 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:54.438959 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41f248e-f176-4f42-8926-1e54ff520d43 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:54.460318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 589/1177] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 23 16:55:54 2022] POST /identity/v3/users => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:55:56.654313 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e949c-986a-4633-a8f0-518f11181e2d None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:56.655192 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e949c-986a-4633-a8f0-518f11181e2d None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:56.655192 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e949c-986a-4633-a8f0-518f11181e2d None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:56.657676 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 589/1178] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:55:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:55:56.663939 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad34f0b-2846-47fe-8e98-21aa676ad30a None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:56.664377 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad34f0b-2846-47fe-8e98-21aa676ad30a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:56.664509 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad34f0b-2846-47fe-8e98-21aa676ad30a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:56.686009 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-3ad34f0b-2846-47fe-8e98-21aa676ad30a None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:55:56.687372 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-3ad34f0b-2846-47fe-8e98-21aa676ad30a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:55:56.760044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 590/1179] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:55:56 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:55:56.784353 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e949c-986a-4633-a8f0-518f11181e2d None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:56.901197 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e949c-986a-4633-a8f0-518f11181e2d None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:56.937794 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4874a28-a5d0-493b-9509-a37d0ddb5d7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:56.938578 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4874a28-a5d0-493b-9509-a37d0ddb5d7d None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:56.938860 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4874a28-a5d0-493b-9509-a37d0ddb5d7d None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:56.939177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4874a28-a5d0-493b-9509-a37d0ddb5d7d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:56.950084 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4874a28-a5d0-493b-9509-a37d0ddb5d7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:55:56.958169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-f4874a28-a5d0-493b-9509-a37d0ddb5d7d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:55:56.958169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:55:56.958169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:55:56.958169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:55:56.958169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:55:56.958169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:55:56.958169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:55:56.958169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:55:56.958169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:55:56.958169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:55:56.958169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:55:56.958169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:55:56.958169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:55:56.958169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:55:56.958169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:55:56.958169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:55:56.959309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:55:56.959309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:55:56.959309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:55:56.959309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:55:56.959309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:55:56.959309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:55:56.959309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:55:56.959309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:55:56.959309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:55:56.959309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:55:56.959309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:55:56.959309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:55:56.959309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:55:56.959309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:55:56.959309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:55:56.959309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:55:56.960375 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:55:56.960375 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:55:56.960375 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:55:56.960375 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:55:56.960375 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:55:56.960375 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:55:56.960375 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:55:56.960375 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:55:56.960375 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:55:56.960375 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:55:56.960375 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:55:56.960375 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:55:56.963091 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 590/1180] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:55:56 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:55:56.965538 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad34f0b-2846-47fe-8e98-21aa676ad30a None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:56.977575 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:57.075063 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad34f0b-2846-47fe-8e98-21aa676ad30a None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:57.110833 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329db27e-bce7-4327-831e-7dca6486b96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:57.111527 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329db27e-bce7-4327-831e-7dca6486b96b None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:57.111809 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329db27e-bce7-4327-831e-7dca6486b96b None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:57.112085 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329db27e-bce7-4327-831e-7dca6486b96b None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:57.112827 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329db27e-bce7-4327-831e-7dca6486b96b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:57.113135 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329db27e-bce7-4327-831e-7dca6486b96b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:57.114524 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329db27e-bce7-4327-831e-7dca6486b96b None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:57.122701 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 591/1181] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:55:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:55:57.129439 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4874a28-a5d0-493b-9509-a37d0ddb5d7d None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:57.146280 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:57.236064 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4874a28-a5d0-493b-9509-a37d0ddb5d7d None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:57.273657 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804aba34-f6b3-434f-a26d-c2ced0488b34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:57.274518 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804aba34-f6b3-434f-a26d-c2ced0488b34 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:57.274795 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804aba34-f6b3-434f-a26d-c2ced0488b34 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:57.275092 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804aba34-f6b3-434f-a26d-c2ced0488b34 None admin] PATH_INFO: `/v3/users/9ab39281b8954509bb1ca6a4d46fab4d` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:57.288649 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804aba34-f6b3-434f-a26d-c2ced0488b34 None admin] RBAC: Authorizing `identity:get_user(user_id=9ab39281b8954509bb1ca6a4d46fab4d)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:57.290094 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804aba34-f6b3-434f-a26d-c2ced0488b34 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:57.291765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 591/1182] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:55:57 2022] GET /identity/v3/users/9ab39281b8954509bb1ca6a4d46fab4d => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:57.298236 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329db27e-bce7-4327-831e-7dca6486b96b None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:57.382761 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:57.417518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329db27e-bce7-4327-831e-7dca6486b96b None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:57.463366 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e06544-cbd9-4b45-ad39-4226200caa41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:57.464012 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e06544-cbd9-4b45-ad39-4226200caa41 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:57.464291 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e06544-cbd9-4b45-ad39-4226200caa41 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:57.464562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e06544-cbd9-4b45-ad39-4226200caa41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:57.542255 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e06544-cbd9-4b45-ad39-4226200caa41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:57.543751 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e06544-cbd9-4b45-ad39-4226200caa41 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:57.612705 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:57.644037 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 592/1183] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:55:57 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 16:55:57.650665 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804aba34-f6b3-434f-a26d-c2ced0488b34 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:57.720231 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:55:57.761251 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804aba34-f6b3-434f-a26d-c2ced0488b34 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:57.797091 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e926854a-e117-4ca6-8b00-4434a285287b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:57.797936 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926854a-e117-4ca6-8b00-4434a285287b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:57.798219 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926854a-e117-4ca6-8b00-4434a285287b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:57.798491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e926854a-e117-4ca6-8b00-4434a285287b None admin] PATH_INFO: `/v3/projects/9dd8c2df627d419c99fdc0fcde5c37e6` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:57.799708 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926854a-e117-4ca6-8b00-4434a285287b None admin] RBAC: Authorizing `identity:get_project(project_id=9dd8c2df627d419c99fdc0fcde5c37e6)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:57.801146 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e926854a-e117-4ca6-8b00-4434a285287b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:57.802825 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 592/1184] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:55:57 2022] GET /identity/v3/projects/9dd8c2df627d419c99fdc0fcde5c37e6 => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:55:57.812538 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e06544-cbd9-4b45-ad39-4226200caa41 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:55:57.927497 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e06544-cbd9-4b45-ad39-4226200caa41 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:55:57.962098 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71a7910-700b-4dc4-9323-8303d8b80176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:55:57.962752 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a7910-700b-4dc4-9323-8303d8b80176 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:55:57.963062 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a7910-700b-4dc4-9323-8303d8b80176 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:55:57.963336 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71a7910-700b-4dc4-9323-8303d8b80176 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:55:57.964164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a7910-700b-4dc4-9323-8303d8b80176 None admin] RBAC: Adding query filter params (role.id=17bbe7cc1c3641509de5e9dca8534102, scope.project.id=9dd8c2df627d419c99fdc0fcde5c37e6, user.id=9ab39281b8954509bb1ca6a4d46fab4d) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:55:57.964474 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a7910-700b-4dc4-9323-8303d8b80176 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:55:57.965920 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71a7910-700b-4dc4-9323-8303d8b80176 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:55:57.971707 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 593/1185] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:55:57 2022] GET /identity/v3/role_assignments?user.id=9ab39281b8954509bb1ca6a4d46fab4d&scope.project.id=9dd8c2df627d419c99fdc0fcde5c37e6&role.id=17bbe7cc1c3641509de5e9dca8534102 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:56:00.177381 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a5fe4-197c-46dd-890a-873239f3133f None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:56:00.177960 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a5fe4-197c-46dd-890a-873239f3133f None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:56:00.177960 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3a5fe4-197c-46dd-890a-873239f3133f None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:56:00.179928 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 593/1186] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:56:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:56:00.188244 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f90506-a898-4faf-b65b-61b0020c1dd0 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:56:00.188720 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f90506-a898-4faf-b65b-61b0020c1dd0 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:56:00.188813 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f90506-a898-4faf-b65b-61b0020c1dd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:56:00.206369 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-e5f90506-a898-4faf-b65b-61b0020c1dd0 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:56:00.207570 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e5f90506-a898-4faf-b65b-61b0020c1dd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:56:00.239222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:56:00.284151 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 594/1187] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:56:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 16:56:00.306703 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a5fe4-197c-46dd-890a-873239f3133f None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:56:00.415367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3a5fe4-197c-46dd-890a-873239f3133f None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:56:00.450504 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd704a-3e54-4fed-b1c2-5ef2157e52f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:56:00.451252 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd704a-3e54-4fed-b1c2-5ef2157e52f8 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:56:00.451526 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd704a-3e54-4fed-b1c2-5ef2157e52f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:56:00.451795 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afd704a-3e54-4fed-b1c2-5ef2157e52f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:56:00.454633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:56:00.463120 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afd704a-3e54-4fed-b1c2-5ef2157e52f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:56:00.471145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-5afd704a-3e54-4fed-b1c2-5ef2157e52f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:56:00.471145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:56:00.471145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:56:00.471145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:56:00.471145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:56:00.471145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:56:00.471145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:56:00.471145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:56:00.471145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:56:00.471145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:56:00.471145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:56:00.471145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:56:00.471145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:56:00.471145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:56:00.471145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:56:00.471145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:56:00.472255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:56:00.472255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:56:00.472255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:56:00.472255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:56:00.472255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:56:00.472255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:56:00.472255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:56:00.472255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:56:00.472255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:56:00.472255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:56:00.472255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:56:00.472255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:56:00.472255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:56:00.472255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:56:00.472255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:56:00.472255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:56:00.473333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:56:00.473333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:56:00.473333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:56:00.473333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:56:00.473333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:56:00.473333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:56:00.473333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:56:00.473333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:56:00.473333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:56:00.473333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:56:00.473333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:56:00.473333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:56:00.473333 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 594/1188] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:56:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:56:00.478654 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f90506-a898-4faf-b65b-61b0020c1dd0 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:56:00.582318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f90506-a898-4faf-b65b-61b0020c1dd0 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:56:00.619997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3909c8-0f22-4da1-ab25-09025a977c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:56:00.620648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3909c8-0f22-4da1-ab25-09025a977c5d None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:56:00.620923 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3909c8-0f22-4da1-ab25-09025a977c5d None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:56:00.621196 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3909c8-0f22-4da1-ab25-09025a977c5d None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:56:00.621926 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3909c8-0f22-4da1-ab25-09025a977c5d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:56:00.622224 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3909c8-0f22-4da1-ab25-09025a977c5d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:56:00.623636 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3909c8-0f22-4da1-ab25-09025a977c5d None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:56:00.632535 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 595/1189] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:56:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:56:00.639422 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd704a-3e54-4fed-b1c2-5ef2157e52f8 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:56:00.755331 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afd704a-3e54-4fed-b1c2-5ef2157e52f8 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:56:00.791160 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415044ff-56be-4e49-b8a3-3dfcdbd706ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:56:00.792046 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415044ff-56be-4e49-b8a3-3dfcdbd706ed None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:56:00.792323 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415044ff-56be-4e49-b8a3-3dfcdbd706ed None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:56:00.792592 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415044ff-56be-4e49-b8a3-3dfcdbd706ed None admin] PATH_INFO: `/v3/users/9ab39281b8954509bb1ca6a4d46fab4d` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:56:00.793953 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415044ff-56be-4e49-b8a3-3dfcdbd706ed None admin] RBAC: Authorizing `identity:get_user(user_id=9ab39281b8954509bb1ca6a4d46fab4d)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:56:00.795447 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415044ff-56be-4e49-b8a3-3dfcdbd706ed None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:56:00.797836 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 595/1190] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:56:00 2022] GET /identity/v3/users/9ab39281b8954509bb1ca6a4d46fab4d => generated 311 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:56:00.807092 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3909c8-0f22-4da1-ab25-09025a977c5d None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:56:00.841343 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:56:00.924958 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3909c8-0f22-4da1-ab25-09025a977c5d None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:56:00.960339 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43194aa1-fcfa-4374-a3ff-c5f1147293da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:56:00.961201 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43194aa1-fcfa-4374-a3ff-c5f1147293da None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:56:00.961484 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43194aa1-fcfa-4374-a3ff-c5f1147293da None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:56:00.961751 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43194aa1-fcfa-4374-a3ff-c5f1147293da None admin] PATH_INFO: `/v3/projects/9dd8c2df627d419c99fdc0fcde5c37e6` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:56:00.963078 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43194aa1-fcfa-4374-a3ff-c5f1147293da None admin] RBAC: Authorizing `identity:get_project(project_id=9dd8c2df627d419c99fdc0fcde5c37e6)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:56:00.964521 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43194aa1-fcfa-4374-a3ff-c5f1147293da None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:56:00.966195 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 596/1191] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:56:00 2022] GET /identity/v3/projects/9dd8c2df627d419c99fdc0fcde5c37e6 => generated 361 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:56:00.972958 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415044ff-56be-4e49-b8a3-3dfcdbd706ed None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:56:01.042464 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:56:01.082520 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415044ff-56be-4e49-b8a3-3dfcdbd706ed None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:56:01.117369 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2316e76d-440d-4fa7-88f8-058d2330118f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:56:01.118254 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316e76d-440d-4fa7-88f8-058d2330118f None admin] REQUEST_METHOD: `PUT` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:56:01.118533 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316e76d-440d-4fa7-88f8-058d2330118f None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:56:01.118806 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2316e76d-440d-4fa7-88f8-058d2330118f None admin] PATH_INFO: `/v3/projects/9dd8c2df627d419c99fdc0fcde5c37e6/users/9ab39281b8954509bb1ca6a4d46fab4d/roles/17bbe7cc1c3641509de5e9dca8534102` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:56:01.121052 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316e76d-440d-4fa7-88f8-058d2330118f None admin] RBAC: Authorizing `identity:create_grant(project_id=9dd8c2df627d419c99fdc0fcde5c37e6, user_id=9ab39281b8954509bb1ca6a4d46fab4d, role_id=17bbe7cc1c3641509de5e9dca8534102)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:56:01.122478 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2316e76d-440d-4fa7-88f8-058d2330118f None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:56:01.139121 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 596/1192] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:56:00 2022] PUT /identity/v3/projects/9dd8c2df627d419c99fdc0fcde5c37e6/users/9ab39281b8954509bb1ca6a4d46fab4d/roles/17bbe7cc1c3641509de5e9dca8534102 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:56:03.369159 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0830a4-89a6-4a9c-8040-26f4034c076d None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:56:03.370004 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0830a4-89a6-4a9c-8040-26f4034c076d None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:56:03.370004 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0830a4-89a6-4a9c-8040-26f4034c076d None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:56:03.372303 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 597/1193] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:56:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:56:03.378872 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662f97a-a525-40f3-8086-43c4508c63cf None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:56:03.379599 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662f97a-a525-40f3-8086-43c4508c63cf None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:56:03.379787 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662f97a-a525-40f3-8086-43c4508c63cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:56:03.401436 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-6662f97a-a525-40f3-8086-43c4508c63cf None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:56:03.402663 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-6662f97a-a525-40f3-8086-43c4508c63cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:56:03.480154 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 597/1194] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:56:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:56:03.509883 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0830a4-89a6-4a9c-8040-26f4034c076d None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:56:03.629433 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:56:03.637542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0830a4-89a6-4a9c-8040-26f4034c076d None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:56:03.672322 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df050e-c218-4b4e-85cf-0ab87f99a4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:56:03.673103 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df050e-c218-4b4e-85cf-0ab87f99a4fa None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:56:03.673380 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df050e-c218-4b4e-85cf-0ab87f99a4fa None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:56:03.673646 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51df050e-c218-4b4e-85cf-0ab87f99a4fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:56:03.684517 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51df050e-c218-4b4e-85cf-0ab87f99a4fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:56:03.692203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-51df050e-c218-4b4e-85cf-0ab87f99a4fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:56:03.692203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:56:03.692203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:56:03.692203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:56:03.692203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:56:03.692203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:56:03.692203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:56:03.692203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:56:03.692203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:56:03.692203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:56:03.692203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:56:03.692203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:56:03.692203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:56:03.692203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:56:03.692203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:56:03.692203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:56:03.693230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:56:03.693230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:56:03.693230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:56:03.693230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:56:03.693230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:56:03.693230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:56:03.693230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:56:03.693230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:56:03.693230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:56:03.693230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:56:03.693230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:56:03.693230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:56:03.693230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:56:03.693230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:56:03.693230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:56:03.693230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:56:03.694233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:56:03.694233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:56:03.694233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:56:03.694233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:56:03.694233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:56:03.694233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:56:03.694233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:56:03.694233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:56:03.694233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:56:03.694233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:56:03.694233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:56:03.694233 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:56:03.698519 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 598/1195] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:56:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:56:03.699648 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662f97a-a525-40f3-8086-43c4508c63cf None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:56:03.765580 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:56:03.838411 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662f97a-a525-40f3-8086-43c4508c63cf None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:56:03.874934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d277586-f90e-42de-9ff2-12d3d684344a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:56:03.875625 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d277586-f90e-42de-9ff2-12d3d684344a None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:56:03.875908 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d277586-f90e-42de-9ff2-12d3d684344a None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:56:03.876182 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d277586-f90e-42de-9ff2-12d3d684344a None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:56:03.876930 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d277586-f90e-42de-9ff2-12d3d684344a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:56:03.877238 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d277586-f90e-42de-9ff2-12d3d684344a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:56:03.878620 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d277586-f90e-42de-9ff2-12d3d684344a None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:56:03.887296 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 598/1196] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:56:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:56:03.896836 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df050e-c218-4b4e-85cf-0ab87f99a4fa None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:56:04.011461 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51df050e-c218-4b4e-85cf-0ab87f99a4fa None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:56:04.046967 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc2164-72e9-4efb-99db-0e960439f7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:56:04.047852 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc2164-72e9-4efb-99db-0e960439f7d4 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:56:04.048143 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc2164-72e9-4efb-99db-0e960439f7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:56:04.048414 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37fc2164-72e9-4efb-99db-0e960439f7d4 None admin] PATH_INFO: `/v3/users/9ab39281b8954509bb1ca6a4d46fab4d` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:56:04.049736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc2164-72e9-4efb-99db-0e960439f7d4 None admin] RBAC: Authorizing `identity:get_user(user_id=9ab39281b8954509bb1ca6a4d46fab4d)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:56:04.051212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37fc2164-72e9-4efb-99db-0e960439f7d4 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:56:04.052832 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 599/1197] 10.222.0.13 () {56 vars in 1177 bytes} [Mon May 23 16:56:03 2022] GET /identity/v3/users/9ab39281b8954509bb1ca6a4d46fab4d => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:56:04.059430 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d277586-f90e-42de-9ff2-12d3d684344a None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:56:04.173949 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d277586-f90e-42de-9ff2-12d3d684344a None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:56:04.190101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:56:04.211777 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78e1e5-30a7-45ed-94fe-d92edc6e62a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:56:04.212424 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78e1e5-30a7-45ed-94fe-d92edc6e62a3 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:56:04.212703 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78e1e5-30a7-45ed-94fe-d92edc6e62a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:56:04.212973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e78e1e5-30a7-45ed-94fe-d92edc6e62a3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:56:04.284444 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e78e1e5-30a7-45ed-94fe-d92edc6e62a3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:56:04.285879 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e78e1e5-30a7-45ed-94fe-d92edc6e62a3 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:56:04.397787 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 599/1198] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:56:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 16:56:04.403974 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc2164-72e9-4efb-99db-0e960439f7d4 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:56:04.432390 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:56:04.514363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37fc2164-72e9-4efb-99db-0e960439f7d4 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:56:04.549314 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4961d-7859-49fd-abe2-96a168858f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:56:04.550170 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4961d-7859-49fd-abe2-96a168858f09 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:56:04.550454 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4961d-7859-49fd-abe2-96a168858f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:56:04.550725 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4961d-7859-49fd-abe2-96a168858f09 None admin] PATH_INFO: `/v3/projects/9dd8c2df627d419c99fdc0fcde5c37e6` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:56:04.551975 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4961d-7859-49fd-abe2-96a168858f09 None admin] RBAC: Authorizing `identity:get_project(project_id=9dd8c2df627d419c99fdc0fcde5c37e6)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:56:04.553416 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80a4961d-7859-49fd-abe2-96a168858f09 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:56:04.555143 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 600/1199] 10.222.0.13 () {56 vars in 1186 bytes} [Mon May 23 16:56:04 2022] GET /identity/v3/projects/9dd8c2df627d419c99fdc0fcde5c37e6 => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:56:04.561563 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78e1e5-30a7-45ed-94fe-d92edc6e62a3 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:56:04.581229 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:56:04.668116 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e78e1e5-30a7-45ed-94fe-d92edc6e62a3 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:56:04.706855 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a69e13-d256-4ae0-a12c-fbf69f8f16f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:56:04.707465 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a69e13-d256-4ae0-a12c-fbf69f8f16f2 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:56:04.707747 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a69e13-d256-4ae0-a12c-fbf69f8f16f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:56:04.708023 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a69e13-d256-4ae0-a12c-fbf69f8f16f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:56:04.708849 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a69e13-d256-4ae0-a12c-fbf69f8f16f2 None admin] RBAC: Adding query filter params (role.id=17bbe7cc1c3641509de5e9dca8534102, scope.project.id=9dd8c2df627d419c99fdc0fcde5c37e6, user.id=9ab39281b8954509bb1ca6a4d46fab4d) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:56:04.709167 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a69e13-d256-4ae0-a12c-fbf69f8f16f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:56:04.710622 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a69e13-d256-4ae0-a12c-fbf69f8f16f2 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:56:04.716701 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 600/1200] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:56:04 2022] GET /identity/v3/role_assignments?user.id=9ab39281b8954509bb1ca6a4d46fab4d&scope.project.id=9dd8c2df627d419c99fdc0fcde5c37e6&role.id=17bbe7cc1c3641509de5e9dca8534102 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:56:14.544773 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=54 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:56:14.713874 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=60 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:04.819451 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf0e78-1987-4113-b178-4466af58563c None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:04.819995 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf0e78-1987-4113-b178-4466af58563c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:04.819995 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fccf0e78-1987-4113-b178-4466af58563c None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:04.821602 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 601/1201] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:57:04.827190 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fae543-0632-4033-bcfa-4407e3236669 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:04.827700 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fae543-0632-4033-bcfa-4407e3236669 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:04.827884 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fae543-0632-4033-bcfa-4407e3236669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:04.849637 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-c5fae543-0632-4033-bcfa-4407e3236669 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:57:04.850842 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-c5fae543-0632-4033-bcfa-4407e3236669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:57:04.935176 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 601/1202] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:57:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:57:04.964957 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf0e78-1987-4113-b178-4466af58563c None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:05.099948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fccf0e78-1987-4113-b178-4466af58563c None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:05.150085 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a7691d-f20d-4cb4-9bfc-14ad71489b9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:05.151139 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7691d-f20d-4cb4-9bfc-14ad71489b9e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:05.151492 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7691d-f20d-4cb4-9bfc-14ad71489b9e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:05.151875 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a7691d-f20d-4cb4-9bfc-14ad71489b9e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:05.168457 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a7691d-f20d-4cb4-9bfc-14ad71489b9e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:05.170163 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69a7691d-f20d-4cb4-9bfc-14ad71489b9e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:05.192548 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-69a7691d-f20d-4cb4-9bfc-14ad71489b9e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:05.192548 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:05.192548 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:05.192548 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:05.192548 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:05.192548 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:05.192548 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:05.192548 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:05.192548 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:05.192548 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:05.192548 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:05.192548 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:05.192548 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:57:05.192548 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:57:05.192548 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:57:05.193736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:57:05.193736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:05.193736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:05.193736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:05.193736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:05.193736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:05.193736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:05.193736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:05.193736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:57:05.193736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:05.193736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:05.193736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:05.193736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:05.193736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:05.193736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:05.193736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:05.193736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:05.194846 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:57:05.194846 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:57:05.194846 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:05.194846 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:57:05.198794 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 602/1203] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:57:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:05.201426 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fae543-0632-4033-bcfa-4407e3236669 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:05.338688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fae543-0632-4033-bcfa-4407e3236669 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:05.389452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:05.396276 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c073dbce-ebed-4f74-b1a4-e82454a25b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:05.397104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073dbce-ebed-4f74-b1a4-e82454a25b58 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:05.397461 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073dbce-ebed-4f74-b1a4-e82454a25b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:05.397669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073dbce-ebed-4f74-b1a4-e82454a25b58 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:05.398582 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c073dbce-ebed-4f74-b1a4-e82454a25b58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:05.399013 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c073dbce-ebed-4f74-b1a4-e82454a25b58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:05.400745 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c073dbce-ebed-4f74-b1a4-e82454a25b58 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:05.419683 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 602/1204] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:57:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:05.421243 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a7691d-f20d-4cb4-9bfc-14ad71489b9e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:05.494446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:05.553293 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a7691d-f20d-4cb4-9bfc-14ad71489b9e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:05.595661 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef431af-6197-4c7c-afc4-33ed6c416956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:05.596369 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef431af-6197-4c7c-afc4-33ed6c416956 None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:05.596653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef431af-6197-4c7c-afc4-33ed6c416956 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:05.596925 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef431af-6197-4c7c-afc4-33ed6c416956 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:05.597626 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef431af-6197-4c7c-afc4-33ed6c416956 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:05.599888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef431af-6197-4c7c-afc4-33ed6c416956 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:05.630264 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 603/1205] 10.222.0.13 () {60 vars in 1132 bytes} [Mon May 23 16:57:05 2022] POST /identity/v3/users => generated 251 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:57:07.996267 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b51716-1eb7-449b-a94c-ac0bea8042ae None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:07.996882 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b51716-1eb7-449b-a94c-ac0bea8042ae None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:07.996882 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b51716-1eb7-449b-a94c-ac0bea8042ae None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:07.998108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 603/1206] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:57:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:57:08.003707 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348aca47-4e5f-41fa-ae04-e2cc164f279e None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:08.004447 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348aca47-4e5f-41fa-ae04-e2cc164f279e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:08.004619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348aca47-4e5f-41fa-ae04-e2cc164f279e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:08.023249 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-348aca47-4e5f-41fa-ae04-e2cc164f279e None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:57:08.024430 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-348aca47-4e5f-41fa-ae04-e2cc164f279e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:57:08.097648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 604/1207] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:57:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:57:08.122021 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b51716-1eb7-449b-a94c-ac0bea8042ae None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:08.242323 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b51716-1eb7-449b-a94c-ac0bea8042ae None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:08.287336 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ead291-9821-4a64-9224-bda283d806a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:08.288069 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ead291-9821-4a64-9224-bda283d806a3 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:08.288355 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ead291-9821-4a64-9224-bda283d806a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:08.288631 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ead291-9821-4a64-9224-bda283d806a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:08.301867 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ead291-9821-4a64-9224-bda283d806a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:57:08.310257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-74ead291-9821-4a64-9224-bda283d806a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 16:57:08.310257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:08.310257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:08.310257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:08.310257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:08.310257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:08.310257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:08.310257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:08.310257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:08.310257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:08.310257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:08.310257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:08.310257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:57:08.310257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:57:08.310257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:57:08.310257 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:57:08.311283 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:57:08.311283 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:57:08.311283 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:57:08.311283 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:57:08.311283 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:08.311283 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:08.311283 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:08.311283 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:08.311283 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:08.311283 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:08.311283 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:08.311283 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:57:08.311283 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:08.311283 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:08.311283 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:08.311283 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:08.312130 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:08.312130 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:08.312130 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:08.312130 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:08.312130 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:57:08.312130 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:57:08.312130 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:57:08.312130 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:57:08.312130 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:57:08.312130 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:57:08.312130 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 16:57:08.312130 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:57:08.312130 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 604/1208] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 23 16:57:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:08.317764 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348aca47-4e5f-41fa-ae04-e2cc164f279e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:08.362329 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:08.434664 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348aca47-4e5f-41fa-ae04-e2cc164f279e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:08.477140 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababa35b-ecf5-48ce-a409-6343f07676f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:08.477981 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababa35b-ecf5-48ce-a409-6343f07676f3 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:08.478230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababa35b-ecf5-48ce-a409-6343f07676f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:08.478503 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababa35b-ecf5-48ce-a409-6343f07676f3 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:08.479283 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababa35b-ecf5-48ce-a409-6343f07676f3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:08.479588 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababa35b-ecf5-48ce-a409-6343f07676f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:08.481251 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ababa35b-ecf5-48ce-a409-6343f07676f3 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:08.490474 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 605/1209] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 23 16:57:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:08.496629 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ead291-9821-4a64-9224-bda283d806a3 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:08.561433 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:08.623141 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ead291-9821-4a64-9224-bda283d806a3 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:08.663872 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e43509-81f0-4e76-a5bc-1e4563313fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:08.664802 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e43509-81f0-4e76-a5bc-1e4563313fcf None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:08.665047 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e43509-81f0-4e76-a5bc-1e4563313fcf None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:08.665380 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e43509-81f0-4e76-a5bc-1e4563313fcf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:08.677491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e43509-81f0-4e76-a5bc-1e4563313fcf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:08.679101 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e43509-81f0-4e76-a5bc-1e4563313fcf None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:08.697682 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-73e43509-81f0-4e76-a5bc-1e4563313fcf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:08.697682 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:08.697682 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:08.697682 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:08.697682 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:08.697682 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:08.697682 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:08.697682 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:08.697682 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:08.697682 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:08.697682 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:08.697682 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:08.697682 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:57:08.697682 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:57:08.697682 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:57:08.703871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:57:08.703871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:08.703871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:08.703871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:08.703871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:08.703871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:08.703871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:08.703871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:08.703871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:57:08.703871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:08.703871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:08.703871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:08.703871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:08.703871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:08.703871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:08.703871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:08.703871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:08.704779 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:57:08.704779 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:57:08.704779 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:08.704779 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:57:08.704779 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 605/1210] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:57:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:08.705077 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababa35b-ecf5-48ce-a409-6343f07676f3 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:08.826783 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:08.829890 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ababa35b-ecf5-48ce-a409-6343f07676f3 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:08.872366 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516c70f-fb1a-4f44-a453-cd6a5bda6a1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:08.873202 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516c70f-fb1a-4f44-a453-cd6a5bda6a1e None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:08.873475 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516c70f-fb1a-4f44-a453-cd6a5bda6a1e None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:08.873750 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c516c70f-fb1a-4f44-a453-cd6a5bda6a1e None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:08.874475 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c516c70f-fb1a-4f44-a453-cd6a5bda6a1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:08.874785 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c516c70f-fb1a-4f44-a453-cd6a5bda6a1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:08.876200 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c516c70f-fb1a-4f44-a453-cd6a5bda6a1e None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:08.894585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 606/1211] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:57:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:08.895995 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e43509-81f0-4e76-a5bc-1e4563313fcf None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:09.016601 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e43509-81f0-4e76-a5bc-1e4563313fcf None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:09.018283 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:09.054186 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823f49b-4148-4571-9b90-6ae48c8ff25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:09.055022 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823f49b-4148-4571-9b90-6ae48c8ff25c None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:09.055303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823f49b-4148-4571-9b90-6ae48c8ff25c None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:09.055571 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6823f49b-4148-4571-9b90-6ae48c8ff25c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:09.065511 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6823f49b-4148-4571-9b90-6ae48c8ff25c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:09.067787 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6823f49b-4148-4571-9b90-6ae48c8ff25c None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:09.089014 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-6823f49b-4148-4571-9b90-6ae48c8ff25c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 16:57:09.089014 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:09.089014 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:09.089014 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:09.089014 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:09.089014 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:09.089014 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:09.089014 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:09.089014 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:09.089014 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:09.089014 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:09.089014 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:09.089014 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:57:09.089014 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:57:09.089014 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:57:09.089014 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:57:09.092955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:09.092955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:09.092955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:57:09.092955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:57:09.092955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:57:09.092955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:57:09.092955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:09.092955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:09.092955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:09.092955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:09.092955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:09.092955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:57:09.092955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:09.092955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:09.092955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:09.092955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:09.092955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:09.092955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:09.093903 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:09.093903 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:09.093903 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:57:09.093903 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:57:09.093903 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:57:09.093903 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:57:09.093903 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:57:09.093903 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:57:09.093903 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 16:57:09.093903 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:57:09.093903 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 606/1212] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:57:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:57:09.096853 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516c70f-fb1a-4f44-a453-cd6a5bda6a1e None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:09.222184 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c516c70f-fb1a-4f44-a453-cd6a5bda6a1e None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:09.266316 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9adc1d-8484-432e-af7c-ca407b92dccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:09.267410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9adc1d-8484-432e-af7c-ca407b92dccb None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:09.267763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9adc1d-8484-432e-af7c-ca407b92dccb None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:09.268182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9adc1d-8484-432e-af7c-ca407b92dccb None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:09.270168 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9adc1d-8484-432e-af7c-ca407b92dccb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:09.270168 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9adc1d-8484-432e-af7c-ca407b92dccb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:09.271417 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9adc1d-8484-432e-af7c-ca407b92dccb None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:09.295505 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 607/1213] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 23 16:57:09 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:09.301339 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823f49b-4148-4571-9b90-6ae48c8ff25c None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:09.418312 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6823f49b-4148-4571-9b90-6ae48c8ff25c None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:09.458434 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7442c-23eb-4a19-8c1c-2b54186acd5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:09.459296 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7442c-23eb-4a19-8c1c-2b54186acd5b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:09.459586 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7442c-23eb-4a19-8c1c-2b54186acd5b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:09.459859 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e7442c-23eb-4a19-8c1c-2b54186acd5b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:09.501415 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:09.545931 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7442c-23eb-4a19-8c1c-2b54186acd5b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:09.547494 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e7442c-23eb-4a19-8c1c-2b54186acd5b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:09.670351 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 607/1214] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:57:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 16:57:09.672834 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9adc1d-8484-432e-af7c-ca407b92dccb None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:09.693176 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:09.785870 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9adc1d-8484-432e-af7c-ca407b92dccb None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:09.833543 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0273d46-52bc-47e7-883d-0bcc507dd139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:09.834357 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0273d46-52bc-47e7-883d-0bcc507dd139 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:09.834637 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0273d46-52bc-47e7-883d-0bcc507dd139 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:09.834930 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0273d46-52bc-47e7-883d-0bcc507dd139 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:09.846193 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0273d46-52bc-47e7-883d-0bcc507dd139 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:09.847772 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0273d46-52bc-47e7-883d-0bcc507dd139 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:09.867029 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-a0273d46-52bc-47e7-883d-0bcc507dd139 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:09.867029 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:09.867029 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:09.867029 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:09.867029 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:09.867029 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:09.867029 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:09.867029 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:09.867029 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:09.867029 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:09.867029 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:09.867029 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:09.867029 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:57:09.867029 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:57:09.867029 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:57:09.868041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:57:09.868041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:09.868041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:09.868041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:09.868041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:09.868041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:09.868041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:09.868041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:09.868041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:57:09.868041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:09.868041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:09.868041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:09.868041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:09.868041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:09.868041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:09.868041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:09.868041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:09.868930 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:57:09.868930 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:57:09.868930 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:09.868930 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:57:09.868930 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 608/1215] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:57:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:09.875481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7442c-23eb-4a19-8c1c-2b54186acd5b None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:09.948017 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:09.996803 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e7442c-23eb-4a19-8c1c-2b54186acd5b None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:10.033650 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6a7d0-a8e7-48df-85a4-3579b1e28a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:10.034474 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6a7d0-a8e7-48df-85a4-3579b1e28a3d None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:10.034863 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6a7d0-a8e7-48df-85a4-3579b1e28a3d None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:10.035250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c6a7d0-a8e7-48df-85a4-3579b1e28a3d None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:10.036063 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c6a7d0-a8e7-48df-85a4-3579b1e28a3d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:10.036465 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c6a7d0-a8e7-48df-85a4-3579b1e28a3d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:10.037936 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c6a7d0-a8e7-48df-85a4-3579b1e28a3d None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:10.050185 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 608/1216] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:57:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:10.056062 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0273d46-52bc-47e7-883d-0bcc507dd139 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:10.138191 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:10.176680 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0273d46-52bc-47e7-883d-0bcc507dd139 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:10.214487 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e2edc-47ed-4194-bfba-fbec4577172c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:10.215258 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e2edc-47ed-4194-bfba-fbec4577172c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:10.215541 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e2edc-47ed-4194-bfba-fbec4577172c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:10.215815 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e2edc-47ed-4194-bfba-fbec4577172c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:10.226255 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e2edc-47ed-4194-bfba-fbec4577172c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:10.228081 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e2edc-47ed-4194-bfba-fbec4577172c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:10.244392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-f96e2edc-47ed-4194-bfba-fbec4577172c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:57:10.244392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:10.244392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:10.244392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:10.244392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:10.244392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:10.244392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:10.244392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:10.244392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:10.244392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:10.244392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:10.244392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:10.244392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:57:10.244392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:57:10.244392 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:57:10.245645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:57:10.245645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:10.245645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:10.245645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:10.245645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:10.245645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:10.245645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:10.245645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:10.245645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:57:10.245645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:10.245645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:10.245645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:10.245645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:10.245645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:10.245645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:10.245645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:10.245645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:10.251766 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:57:10.251766 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:57:10.251766 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:57:10.251766 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:57:10.251766 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:57:10.251766 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:57:10.251766 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:57:10.251766 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:57:10.251766 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 609/1217] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:57:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:10.252430 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6a7d0-a8e7-48df-85a4-3579b1e28a3d None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:10.369669 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c6a7d0-a8e7-48df-85a4-3579b1e28a3d None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:10.397156 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:10.411016 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d06c49c-3dca-443f-ace4-d197d4f65783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:10.412307 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06c49c-3dca-443f-ace4-d197d4f65783 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:10.412307 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06c49c-3dca-443f-ace4-d197d4f65783 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:10.412307 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d06c49c-3dca-443f-ace4-d197d4f65783 None admin] PATH_INFO: `/v3/projects` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:10.413103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06c49c-3dca-443f-ace4-d197d4f65783 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:10.413409 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06c49c-3dca-443f-ace4-d197d4f65783 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:10.414843 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d06c49c-3dca-443f-ace4-d197d4f65783 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:10.433338 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 609/1218] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:57:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:10.434549 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e2edc-47ed-4194-bfba-fbec4577172c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:10.556967 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e2edc-47ed-4194-bfba-fbec4577172c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:10.589882 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:10.597801 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805484d9-8b78-4383-b42d-b74a18271ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:10.598702 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805484d9-8b78-4383-b42d-b74a18271ecd None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:10.599104 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805484d9-8b78-4383-b42d-b74a18271ecd None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:10.599367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805484d9-8b78-4383-b42d-b74a18271ecd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:10.600285 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805484d9-8b78-4383-b42d-b74a18271ecd None admin] RBAC: Adding query filter params (role.id=5a7214dc069f45969e19c44122b378a4, scope.project.id=0595667fb2e04a3ca4716f669d6698b0, user.id=f181ebdfcd544b54af7e6f9c118fe252) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:10.600625 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805484d9-8b78-4383-b42d-b74a18271ecd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:10.602333 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805484d9-8b78-4383-b42d-b74a18271ecd None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:10.608827 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 610/1219] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:57:10 2022] GET /identity/v3/role_assignments?user.id=f181ebdfcd544b54af7e6f9c118fe252&scope.project.id=0595667fb2e04a3ca4716f669d6698b0&role.id=5a7214dc069f45969e19c44122b378a4 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:12.835598 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b11611-8e89-405d-8d20-be7063de2438 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:12.836163 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b11611-8e89-405d-8d20-be7063de2438 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:12.836163 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b11611-8e89-405d-8d20-be7063de2438 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:12.837721 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 610/1220] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:57:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:57:12.844283 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837aef77-aec0-4f2b-8b90-11c9879cb531 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:12.844888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837aef77-aec0-4f2b-8b90-11c9879cb531 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:12.844888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837aef77-aec0-4f2b-8b90-11c9879cb531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:12.862869 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-837aef77-aec0-4f2b-8b90-11c9879cb531 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:57:12.864075 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-837aef77-aec0-4f2b-8b90-11c9879cb531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:57:12.949350 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 611/1221] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:57:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:57:12.972309 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b11611-8e89-405d-8d20-be7063de2438 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:13.100800 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b11611-8e89-405d-8d20-be7063de2438 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:13.145180 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b8954f-2dd0-40dc-92af-4a0b3b09aaa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:13.146172 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b8954f-2dd0-40dc-92af-4a0b3b09aaa4 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:13.146575 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b8954f-2dd0-40dc-92af-4a0b3b09aaa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:13.146854 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b8954f-2dd0-40dc-92af-4a0b3b09aaa4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:13.159311 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b8954f-2dd0-40dc-92af-4a0b3b09aaa4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:57:13.169553 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-12b8954f-2dd0-40dc-92af-4a0b3b09aaa4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 16:57:13.169553 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:13.169553 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:13.169553 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:13.169553 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:13.169553 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:13.169553 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:13.169553 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:13.169553 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:13.169553 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:13.169553 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:13.169553 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:13.169553 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:57:13.169553 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:57:13.169553 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:57:13.169553 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:57:13.170694 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:57:13.170694 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:57:13.170694 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:57:13.170694 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:57:13.170694 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:13.170694 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:13.170694 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:13.170694 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:13.170694 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:13.170694 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:13.170694 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:13.170694 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:57:13.170694 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:13.170694 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:13.170694 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:13.170694 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:13.175978 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:13.175978 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:13.175978 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:13.175978 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:13.175978 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:57:13.175978 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:57:13.175978 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:57:13.175978 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:57:13.175978 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:57:13.175978 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:57:13.175978 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 16:57:13.175978 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:57:13.175978 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 611/1222] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 23 16:57:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:13.176861 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837aef77-aec0-4f2b-8b90-11c9879cb531 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:13.288218 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837aef77-aec0-4f2b-8b90-11c9879cb531 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:13.315120 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:13.326577 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237126ee-9fca-4dda-a948-2469ecd6aaa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:13.327290 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237126ee-9fca-4dda-a948-2469ecd6aaa5 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:13.327574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237126ee-9fca-4dda-a948-2469ecd6aaa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:13.327850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237126ee-9fca-4dda-a948-2469ecd6aaa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:13.328602 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237126ee-9fca-4dda-a948-2469ecd6aaa5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:13.328908 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237126ee-9fca-4dda-a948-2469ecd6aaa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:13.330299 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237126ee-9fca-4dda-a948-2469ecd6aaa5 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:13.338450 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 612/1223] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 23 16:57:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:13.344817 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b8954f-2dd0-40dc-92af-4a0b3b09aaa4 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:13.393621 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:13.456206 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b8954f-2dd0-40dc-92af-4a0b3b09aaa4 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:13.492299 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5b6cd-ca5c-4547-8607-981833f4f66e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:13.493103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5b6cd-ca5c-4547-8607-981833f4f66e None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:13.493386 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5b6cd-ca5c-4547-8607-981833f4f66e None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:13.493660 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbf5b6cd-ca5c-4547-8607-981833f4f66e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:13.505692 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf5b6cd-ca5c-4547-8607-981833f4f66e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:13.507215 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbf5b6cd-ca5c-4547-8607-981833f4f66e None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:13.524917 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-fbf5b6cd-ca5c-4547-8607-981833f4f66e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:13.524917 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:13.524917 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:13.524917 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:13.524917 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:13.524917 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:13.524917 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:13.524917 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:13.524917 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:13.524917 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:13.524917 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:13.524917 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:13.524917 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:57:13.524917 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:57:13.524917 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:57:13.525992 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:57:13.525992 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:13.525992 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:13.525992 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:13.525992 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:13.525992 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:13.525992 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:13.525992 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:13.525992 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:57:13.525992 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:13.525992 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:13.525992 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:13.525992 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:13.525992 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:13.525992 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:13.525992 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:13.525992 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:13.527307 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:57:13.527307 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:57:13.527307 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:13.527307 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:57:13.527307 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 612/1224] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:57:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:13.532567 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237126ee-9fca-4dda-a948-2469ecd6aaa5 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:13.639609 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237126ee-9fca-4dda-a948-2469ecd6aaa5 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:13.673997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba1684-bd39-4b3d-ab66-9b49eb864167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:13.674581 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba1684-bd39-4b3d-ab66-9b49eb864167 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:13.674877 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba1684-bd39-4b3d-ab66-9b49eb864167 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:13.675161 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba1684-bd39-4b3d-ab66-9b49eb864167 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:13.675880 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ba1684-bd39-4b3d-ab66-9b49eb864167 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:13.676185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ba1684-bd39-4b3d-ab66-9b49eb864167 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:13.677631 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ba1684-bd39-4b3d-ab66-9b49eb864167 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:13.690244 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 613/1225] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:13.696957 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5b6cd-ca5c-4547-8607-981833f4f66e None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:13.793157 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:13.800526 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbf5b6cd-ca5c-4547-8607-981833f4f66e None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:13.834194 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdbcf08-a083-49a6-ae31-22bfe09142e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:13.835024 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdbcf08-a083-49a6-ae31-22bfe09142e4 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:13.835310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdbcf08-a083-49a6-ae31-22bfe09142e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:13.835583 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdbcf08-a083-49a6-ae31-22bfe09142e4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:13.843430 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdbcf08-a083-49a6-ae31-22bfe09142e4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:13.844876 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdbcf08-a083-49a6-ae31-22bfe09142e4 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:13.859422 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-fbdbcf08-a083-49a6-ae31-22bfe09142e4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 16:57:13.859422 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:13.859422 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:13.859422 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:13.859422 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:13.859422 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:13.859422 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:13.859422 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:13.859422 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:13.859422 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:13.859422 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:13.859422 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:13.859422 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:57:13.859422 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:57:13.859422 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:57:13.859422 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:57:13.860525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:13.860525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:13.860525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:57:13.860525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:57:13.860525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:57:13.860525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:57:13.860525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:13.860525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:13.860525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:13.860525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:13.860525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:13.860525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:57:13.860525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:13.860525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:13.860525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:13.860525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:13.860525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:13.860525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:13.861492 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:13.861492 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:13.861492 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:57:13.861492 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:57:13.861492 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:57:13.861492 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:57:13.861492 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:57:13.861492 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:57:13.861492 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 16:57:13.861492 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:57:13.861492 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 613/1226] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:57:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:13.866591 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba1684-bd39-4b3d-ab66-9b49eb864167 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:13.915948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:13.978587 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba1684-bd39-4b3d-ab66-9b49eb864167 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:14.015862 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3d0f0-6b7e-43eb-9d5d-0cf3760b15bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:14.016558 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3d0f0-6b7e-43eb-9d5d-0cf3760b15bf None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:14.016839 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3d0f0-6b7e-43eb-9d5d-0cf3760b15bf None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:14.017111 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d3d0f0-6b7e-43eb-9d5d-0cf3760b15bf None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:14.017944 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3d0f0-6b7e-43eb-9d5d-0cf3760b15bf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:14.018248 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3d0f0-6b7e-43eb-9d5d-0cf3760b15bf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:14.019710 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d3d0f0-6b7e-43eb-9d5d-0cf3760b15bf None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:14.037888 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 614/1227] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 23 16:57:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:14.044255 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdbcf08-a083-49a6-ae31-22bfe09142e4 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:14.154010 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdbcf08-a083-49a6-ae31-22bfe09142e4 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:14.195819 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b82a2-0471-443e-ad90-c7235ed1a8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:14.196861 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b82a2-0471-443e-ad90-c7235ed1a8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:14.197143 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b82a2-0471-443e-ad90-c7235ed1a8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:14.197413 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616b82a2-0471-443e-ad90-c7235ed1a8d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:14.220414 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b82a2-0471-443e-ad90-c7235ed1a8d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:14.222173 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616b82a2-0471-443e-ad90-c7235ed1a8d0 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:14.243293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-616b82a2-0471-443e-ad90-c7235ed1a8d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:14.243293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:14.243293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:14.243293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:14.243293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:14.243293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:14.243293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:14.243293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:14.243293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:14.243293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:14.243293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:14.243293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:14.243293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:57:14.243293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:57:14.243293 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:57:14.244352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:57:14.244352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:14.244352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:14.244352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:14.244352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:14.244352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:14.244352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:14.244352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:14.244352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:57:14.244352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:14.244352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:14.244352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:14.244352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:14.244352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:14.244352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:14.244352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:14.244352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:14.245902 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:57:14.245902 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:57:14.245902 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:14.245902 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:57:14.245902 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 614/1228] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:57:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:57:14.250886 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3d0f0-6b7e-43eb-9d5d-0cf3760b15bf None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:14.370856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d3d0f0-6b7e-43eb-9d5d-0cf3760b15bf None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:14.373303 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:14.411546 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9b2d1-5b25-4590-bbca-1c02db225797 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:14.412263 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9b2d1-5b25-4590-bbca-1c02db225797 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:14.412596 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9b2d1-5b25-4590-bbca-1c02db225797 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:14.412882 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab9b2d1-5b25-4590-bbca-1c02db225797 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:14.413701 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab9b2d1-5b25-4590-bbca-1c02db225797 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:14.414011 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab9b2d1-5b25-4590-bbca-1c02db225797 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:14.415513 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab9b2d1-5b25-4590-bbca-1c02db225797 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:14.427724 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 615/1229] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:57:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:14.433588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b82a2-0471-443e-ad90-c7235ed1a8d0 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:14.444783 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:14.549134 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616b82a2-0471-443e-ad90-c7235ed1a8d0 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:14.587441 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ddb8cc-0e26-47d6-aae0-4639982b3712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:14.588295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ddb8cc-0e26-47d6-aae0-4639982b3712 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:14.588577 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ddb8cc-0e26-47d6-aae0-4639982b3712 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:14.589187 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ddb8cc-0e26-47d6-aae0-4639982b3712 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:14.596513 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ddb8cc-0e26-47d6-aae0-4639982b3712 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:14.598035 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ddb8cc-0e26-47d6-aae0-4639982b3712 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:14.613354 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-b9ddb8cc-0e26-47d6-aae0-4639982b3712 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:57:14.613354 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:14.613354 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:14.613354 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:14.613354 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:14.613354 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:14.613354 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:14.613354 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:14.613354 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:14.613354 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:14.613354 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:14.613354 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:14.613354 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:57:14.613354 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:57:14.613354 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:57:14.614637 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:57:14.614637 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:14.614637 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:14.614637 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:14.614637 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:14.614637 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:14.614637 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:14.614637 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:14.614637 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:57:14.614637 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:14.614637 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:14.614637 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:14.614637 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:14.614637 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:14.614637 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:14.614637 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:14.614637 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:14.615826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:57:14.615826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:57:14.615826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:57:14.615826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:57:14.615826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:57:14.615826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:57:14.615826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:57:14.615826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:57:14.615826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 615/1230] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:57:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:14.621486 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9b2d1-5b25-4590-bbca-1c02db225797 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:14.734709 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab9b2d1-5b25-4590-bbca-1c02db225797 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:14.769824 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0915436b-edd2-4c44-ae55-b7ebd405ffe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:14.770488 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915436b-edd2-4c44-ae55-b7ebd405ffe8 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:14.770769 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915436b-edd2-4c44-ae55-b7ebd405ffe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:14.771074 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0915436b-edd2-4c44-ae55-b7ebd405ffe8 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:14.771854 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915436b-edd2-4c44-ae55-b7ebd405ffe8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:14.772162 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915436b-edd2-4c44-ae55-b7ebd405ffe8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:14.773602 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0915436b-edd2-4c44-ae55-b7ebd405ffe8 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:14.786395 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 616/1231] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:57:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:14.793557 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ddb8cc-0e26-47d6-aae0-4639982b3712 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:14.861956 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:14.899212 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ddb8cc-0e26-47d6-aae0-4639982b3712 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:14.935230 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7405cebc-32ce-44ad-a523-3ab023a569c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:14.936120 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405cebc-32ce-44ad-a523-3ab023a569c0 None admin] REQUEST_METHOD: `PUT` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:14.936397 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405cebc-32ce-44ad-a523-3ab023a569c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:14.936670 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7405cebc-32ce-44ad-a523-3ab023a569c0 None admin] PATH_INFO: `/v3/projects/0595667fb2e04a3ca4716f669d6698b0/users/f181ebdfcd544b54af7e6f9c118fe252/roles/5a7214dc069f45969e19c44122b378a4` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:14.952288 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405cebc-32ce-44ad-a523-3ab023a569c0 None admin] RBAC: Authorizing `identity:create_grant(project_id=0595667fb2e04a3ca4716f669d6698b0, user_id=f181ebdfcd544b54af7e6f9c118fe252, role_id=5a7214dc069f45969e19c44122b378a4)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:14.953719 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7405cebc-32ce-44ad-a523-3ab023a569c0 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:14.967734 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 616/1232] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:57:14 2022] PUT /identity/v3/projects/0595667fb2e04a3ca4716f669d6698b0/users/f181ebdfcd544b54af7e6f9c118fe252/roles/5a7214dc069f45969e19c44122b378a4 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:57:17.174601 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3cfeeb-edc0-40d6-9ad9-13664c012aba None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:17.175198 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3cfeeb-edc0-40d6-9ad9-13664c012aba None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:17.175198 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3cfeeb-edc0-40d6-9ad9-13664c012aba None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:17.176704 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 617/1233] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:57:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:57:17.183171 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0baee-5fb7-4559-820e-67bc103b35f4 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:17.183627 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0baee-5fb7-4559-820e-67bc103b35f4 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:17.183938 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c0baee-5fb7-4559-820e-67bc103b35f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:17.201776 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-b7c0baee-5fb7-4559-820e-67bc103b35f4 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:57:17.203045 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-b7c0baee-5fb7-4559-820e-67bc103b35f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:57:17.278512 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 617/1234] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:57:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:57:17.304424 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3cfeeb-edc0-40d6-9ad9-13664c012aba None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:17.337263 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:17.416879 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3cfeeb-edc0-40d6-9ad9-13664c012aba None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:17.452905 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042be97-51f2-4c63-bf02-d5473138c128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:17.453717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042be97-51f2-4c63-bf02-d5473138c128 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:17.453994 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042be97-51f2-4c63-bf02-d5473138c128 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:17.454260 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1042be97-51f2-4c63-bf02-d5473138c128 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:17.464820 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1042be97-51f2-4c63-bf02-d5473138c128 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:57:17.472950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-1042be97-51f2-4c63-bf02-d5473138c128 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 16:57:17.472950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:17.472950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:17.472950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:17.472950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:17.472950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:17.472950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:17.472950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:17.472950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:17.472950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:17.472950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:17.472950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:17.472950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:57:17.472950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:57:17.472950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:57:17.472950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:57:17.475212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:57:17.475212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:57:17.475212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:57:17.475212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:57:17.475212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:17.475212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:17.475212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:17.475212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:17.475212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:17.475212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:17.475212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:17.475212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:57:17.475212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:17.475212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:17.475212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:17.475212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:17.476775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:17.476775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:17.476775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:17.476775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:17.476775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:57:17.476775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:57:17.476775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:57:17.476775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:57:17.476775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:57:17.476775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:57:17.476775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 16:57:17.476775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:57:17.476775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 618/1235] 10.222.0.13 () {56 vars in 1102 bytes} [Mon May 23 16:57:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:17.480808 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0baee-5fb7-4559-820e-67bc103b35f4 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:17.531497 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:17.589040 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c0baee-5fb7-4559-820e-67bc103b35f4 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:17.625820 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637af4a9-3b72-4c36-895d-ef392b06c34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:17.626688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637af4a9-3b72-4c36-895d-ef392b06c34a None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:17.627048 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637af4a9-3b72-4c36-895d-ef392b06c34a None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:17.627361 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637af4a9-3b72-4c36-895d-ef392b06c34a None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:17.628217 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637af4a9-3b72-4c36-895d-ef392b06c34a None admin] RBAC: Adding query filter params (name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:17.628875 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637af4a9-3b72-4c36-895d-ef392b06c34a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:17.630377 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637af4a9-3b72-4c36-895d-ef392b06c34a None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:17.640245 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 618/1236] 10.222.0.13 () {56 vars in 1103 bytes} [Mon May 23 16:57:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:17.646168 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042be97-51f2-4c63-bf02-d5473138c128 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:17.737040 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:17.753840 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1042be97-51f2-4c63-bf02-d5473138c128 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:17.789065 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e853e-f8f5-4b9b-9bff-98a9a24fe74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:17.789855 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e853e-f8f5-4b9b-9bff-98a9a24fe74c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:17.790131 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e853e-f8f5-4b9b-9bff-98a9a24fe74c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:17.790397 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6e853e-f8f5-4b9b-9bff-98a9a24fe74c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:17.800994 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e853e-f8f5-4b9b-9bff-98a9a24fe74c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:17.802457 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6e853e-f8f5-4b9b-9bff-98a9a24fe74c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:17.819935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-ed6e853e-f8f5-4b9b-9bff-98a9a24fe74c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:17.819935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:17.819935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:17.819935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:17.819935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:17.819935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:17.819935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:17.819935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:17.819935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:17.819935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:17.819935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:17.819935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:17.819935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:57:17.819935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:57:17.819935 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:57:17.821025 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:57:17.821025 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:17.821025 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:17.821025 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:17.821025 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:17.821025 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:17.821025 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:17.821025 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:17.821025 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:57:17.821025 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:17.821025 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:17.821025 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:17.821025 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:17.821025 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:17.821025 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:17.821025 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:17.821025 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:17.827647 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:57:17.827647 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:57:17.827647 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:17.827647 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:57:17.827647 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 619/1237] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:57:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:57:17.828001 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637af4a9-3b72-4c36-895d-ef392b06c34a None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:17.933488 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637af4a9-3b72-4c36-895d-ef392b06c34a None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:17.943481 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:17.976131 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7cfd2-24dd-42b9-8514-37835c4b9efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:17.976887 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7cfd2-24dd-42b9-8514-37835c4b9efe None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:17.977224 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7cfd2-24dd-42b9-8514-37835c4b9efe None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:17.977528 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf7cfd2-24dd-42b9-8514-37835c4b9efe None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:17.978316 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7cfd2-24dd-42b9-8514-37835c4b9efe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:17.978613 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7cfd2-24dd-42b9-8514-37835c4b9efe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:17.980596 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf7cfd2-24dd-42b9-8514-37835c4b9efe None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:17.993967 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 619/1238] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:57:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:17.999829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e853e-f8f5-4b9b-9bff-98a9a24fe74c None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:18.110005 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6e853e-f8f5-4b9b-9bff-98a9a24fe74c None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:18.146994 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b142d952-6dfa-40a6-8f87-9254e92c35fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:18.147812 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142d952-6dfa-40a6-8f87-9254e92c35fe None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:18.148096 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142d952-6dfa-40a6-8f87-9254e92c35fe None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:18.148367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b142d952-6dfa-40a6-8f87-9254e92c35fe None admin] PATH_INFO: `/v3/users/placement` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:18.151709 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:18.158663 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b142d952-6dfa-40a6-8f87-9254e92c35fe None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:18.160564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b142d952-6dfa-40a6-8f87-9254e92c35fe None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:18.177050 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-b142d952-6dfa-40a6-8f87-9254e92c35fe None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 16:57:18.177050 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:18.177050 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:18.177050 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:18.177050 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:18.177050 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:18.177050 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:18.177050 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:18.177050 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:18.177050 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:18.177050 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:18.177050 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:18.177050 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:57:18.177050 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:57:18.177050 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:57:18.177050 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:57:18.178657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:18.178657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:18.178657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:57:18.178657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:57:18.178657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:57:18.178657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:57:18.178657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:18.178657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:18.178657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:18.178657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:18.178657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:18.178657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:57:18.178657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:18.178657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:18.178657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:18.178657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:18.178657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:18.178657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:18.179735 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:18.179735 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:18.179735 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:57:18.179735 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:57:18.179735 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:57:18.179735 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:57:18.179735 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:57:18.179735 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:57:18.179735 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 16:57:18.179735 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:57:18.179735 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 620/1239] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:57:17 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:18.184343 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7cfd2-24dd-42b9-8514-37835c4b9efe None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:18.297057 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf7cfd2-24dd-42b9-8514-37835c4b9efe None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:18.338935 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5221c0-0425-4f7a-9b34-b09533c72cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:18.339782 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5221c0-0425-4f7a-9b34-b09533c72cdf None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:18.340064 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5221c0-0425-4f7a-9b34-b09533c72cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:18.340336 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5221c0-0425-4f7a-9b34-b09533c72cdf None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:18.341165 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5221c0-0425-4f7a-9b34-b09533c72cdf None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:18.341471 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5221c0-0425-4f7a-9b34-b09533c72cdf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:18.342893 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5221c0-0425-4f7a-9b34-b09533c72cdf None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:18.365671 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 620/1240] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 23 16:57:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:18.371708 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b142d952-6dfa-40a6-8f87-9254e92c35fe None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:18.489298 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b142d952-6dfa-40a6-8f87-9254e92c35fe None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:18.524563 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea7478-d158-4154-9751-ba44a8ff909a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:18.525517 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea7478-d158-4154-9751-ba44a8ff909a None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:18.525801 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea7478-d158-4154-9751-ba44a8ff909a None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:18.526148 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeea7478-d158-4154-9751-ba44a8ff909a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:18.590816 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:18.599057 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeea7478-d158-4154-9751-ba44a8ff909a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:18.600829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeea7478-d158-4154-9751-ba44a8ff909a None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:18.712912 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 621/1241] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:57:18 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 16:57:18.719167 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5221c0-0425-4f7a-9b34-b09533c72cdf None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:18.748526 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:18.832178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5221c0-0425-4f7a-9b34-b09533c72cdf None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:18.873793 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b04ab2-0d71-457a-88cb-0eba82bc5099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:18.874521 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b04ab2-0d71-457a-88cb-0eba82bc5099 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:18.874800 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b04ab2-0d71-457a-88cb-0eba82bc5099 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:18.875104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b04ab2-0d71-457a-88cb-0eba82bc5099 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:18.886609 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b04ab2-0d71-457a-88cb-0eba82bc5099 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:18.888287 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b04ab2-0d71-457a-88cb-0eba82bc5099 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:18.906732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-67b04ab2-0d71-457a-88cb-0eba82bc5099 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:18.906732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:18.906732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:18.906732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:18.906732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:18.906732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:18.906732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:18.906732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:18.906732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:18.906732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:18.906732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:18.906732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:18.906732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:57:18.906732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:57:18.906732 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:57:18.907870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:57:18.907870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:18.907870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:18.907870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:18.907870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:18.907870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:18.907870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:18.907870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:18.907870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:57:18.907870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:18.907870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:18.907870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:18.907870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:18.907870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:18.907870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:18.907870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:18.907870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:18.908796 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:57:18.908796 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:57:18.908796 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:18.908796 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:57:18.908796 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 621/1242] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:57:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:18.914134 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea7478-d158-4154-9751-ba44a8ff909a None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:19.018349 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:19.024718 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeea7478-d158-4154-9751-ba44a8ff909a None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:19.061670 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105366bd-5f46-411a-9c43-0bd2447c8b68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:19.062361 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105366bd-5f46-411a-9c43-0bd2447c8b68 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:19.062636 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105366bd-5f46-411a-9c43-0bd2447c8b68 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:19.062931 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105366bd-5f46-411a-9c43-0bd2447c8b68 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:19.063712 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105366bd-5f46-411a-9c43-0bd2447c8b68 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:19.064022 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105366bd-5f46-411a-9c43-0bd2447c8b68 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:19.065477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105366bd-5f46-411a-9c43-0bd2447c8b68 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:19.077691 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 622/1243] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:57:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:57:19.083782 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b04ab2-0d71-457a-88cb-0eba82bc5099 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:19.169647 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:19.193534 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b04ab2-0d71-457a-88cb-0eba82bc5099 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:19.228482 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9df0f-978b-411f-8143-cdcf0fb751b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:19.229387 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9df0f-978b-411f-8143-cdcf0fb751b9 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:19.229651 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9df0f-978b-411f-8143-cdcf0fb751b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:19.229973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9df0f-978b-411f-8143-cdcf0fb751b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:19.237382 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9df0f-978b-411f-8143-cdcf0fb751b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:19.238823 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9df0f-978b-411f-8143-cdcf0fb751b9 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:19.253108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-9db9df0f-978b-411f-8143-cdcf0fb751b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:57:19.253108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:19.253108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:19.253108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:19.253108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:19.253108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:19.253108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:19.253108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:19.253108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:19.253108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:19.253108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:19.253108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:19.253108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:57:19.253108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:57:19.253108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:57:19.254709 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:57:19.254709 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:19.254709 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:19.254709 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:19.254709 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:19.254709 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:19.254709 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:19.254709 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:19.254709 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:57:19.254709 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:19.254709 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:19.254709 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:19.254709 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:19.254709 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:19.254709 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:19.254709 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:19.254709 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:19.256603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:57:19.256603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:57:19.256603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:57:19.256603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:57:19.256603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:57:19.256603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:57:19.256603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:57:19.256603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:57:19.256603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 622/1244] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:57:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:57:19.261189 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105366bd-5f46-411a-9c43-0bd2447c8b68 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:19.386808 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105366bd-5f46-411a-9c43-0bd2447c8b68 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:19.422684 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220565b0-2e4c-4d60-93a4-286095f96008 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:19.423318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220565b0-2e4c-4d60-93a4-286095f96008 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:19.423600 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220565b0-2e4c-4d60-93a4-286095f96008 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:19.423909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220565b0-2e4c-4d60-93a4-286095f96008 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:19.424653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220565b0-2e4c-4d60-93a4-286095f96008 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:19.424961 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220565b0-2e4c-4d60-93a4-286095f96008 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:19.426397 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220565b0-2e4c-4d60-93a4-286095f96008 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:19.438829 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 623/1245] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:57:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:19.445417 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9df0f-978b-411f-8143-cdcf0fb751b9 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:19.558587 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9df0f-978b-411f-8143-cdcf0fb751b9 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:19.597452 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb07cac3-59a4-4764-bbfa-d8531c512484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:19.598249 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07cac3-59a4-4764-bbfa-d8531c512484 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:19.598558 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07cac3-59a4-4764-bbfa-d8531c512484 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:19.598874 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb07cac3-59a4-4764-bbfa-d8531c512484 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:19.599866 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07cac3-59a4-4764-bbfa-d8531c512484 None admin] RBAC: Adding query filter params (role.id=5a7214dc069f45969e19c44122b378a4, scope.project.id=0595667fb2e04a3ca4716f669d6698b0, user.id=f181ebdfcd544b54af7e6f9c118fe252) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:19.600233 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07cac3-59a4-4764-bbfa-d8531c512484 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:19.601955 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb07cac3-59a4-4764-bbfa-d8531c512484 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:19.608731 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 623/1246] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:57:19 2022] GET /identity/v3/role_assignments?user.id=f181ebdfcd544b54af7e6f9c118fe252&scope.project.id=0595667fb2e04a3ca4716f669d6698b0&role.id=5a7214dc069f45969e19c44122b378a4 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:22.128926 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfc874-0f45-491d-a046-804459cb26ca None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:22.129538 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfc874-0f45-491d-a046-804459cb26ca None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:22.129538 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29bfc874-0f45-491d-a046-804459cb26ca None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:22.130832 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 624/1247] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:57:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:57:22.137893 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98ea22-14f5-4c5b-9b3a-d4da4bdadc2f None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:22.138306 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98ea22-14f5-4c5b-9b3a-d4da4bdadc2f None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:22.138658 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be98ea22-14f5-4c5b-9b3a-d4da4bdadc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:22.144958 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:22.158785 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-be98ea22-14f5-4c5b-9b3a-d4da4bdadc2f None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:57:22.160203 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-be98ea22-14f5-4c5b-9b3a-d4da4bdadc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:57:22.244441 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 624/1248] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:57:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:57:22.268334 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfc874-0f45-491d-a046-804459cb26ca None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:22.291543 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:22.390021 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29bfc874-0f45-491d-a046-804459cb26ca None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:22.427439 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1f751-f88f-4e10-a328-52e79abca425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:22.428222 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1f751-f88f-4e10-a328-52e79abca425 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:22.428498 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1f751-f88f-4e10-a328-52e79abca425 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:22.428767 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d1f751-f88f-4e10-a328-52e79abca425 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:22.443329 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d1f751-f88f-4e10-a328-52e79abca425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:57:22.450842 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-01d1f751-f88f-4e10-a328-52e79abca425 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:57:22.450842 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:22.450842 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:22.450842 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:22.450842 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:22.450842 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:22.450842 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:22.450842 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:22.450842 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:22.450842 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:22.450842 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:22.450842 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:22.450842 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:57:22.450842 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:57:22.450842 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:57:22.450842 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:57:22.451999 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:57:22.451999 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:57:22.451999 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:57:22.451999 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:57:22.451999 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:22.451999 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:22.451999 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:22.451999 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:22.451999 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:22.451999 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:22.451999 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:22.451999 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:57:22.451999 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:22.451999 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:22.451999 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:22.451999 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:22.452930 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:22.452930 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:22.452930 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:22.452930 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:22.452930 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:57:22.452930 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:57:22.452930 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:57:22.452930 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:57:22.452930 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:57:22.452930 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:57:22.452930 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:57:22.452930 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:57:22.452930 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 625/1249] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:57:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:22.458315 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98ea22-14f5-4c5b-9b3a-d4da4bdadc2f None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:22.567743 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be98ea22-14f5-4c5b-9b3a-d4da4bdadc2f None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:22.605793 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:22.611057 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32888c18-92be-4758-acfa-80f0b3d8249b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:22.611944 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32888c18-92be-4758-acfa-80f0b3d8249b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:22.612276 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32888c18-92be-4758-acfa-80f0b3d8249b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:22.612781 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32888c18-92be-4758-acfa-80f0b3d8249b None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:22.613525 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32888c18-92be-4758-acfa-80f0b3d8249b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:22.613831 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32888c18-92be-4758-acfa-80f0b3d8249b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:22.615237 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32888c18-92be-4758-acfa-80f0b3d8249b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:22.623887 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 625/1250] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:57:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:22.629915 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1f751-f88f-4e10-a328-52e79abca425 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:22.714730 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:22.740139 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d1f751-f88f-4e10-a328-52e79abca425 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:22.777023 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ace8d5-c13e-4dca-975b-acab59622990 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:22.777835 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ace8d5-c13e-4dca-975b-acab59622990 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:22.778120 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ace8d5-c13e-4dca-975b-acab59622990 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:22.778394 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ace8d5-c13e-4dca-975b-acab59622990 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:22.789207 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ace8d5-c13e-4dca-975b-acab59622990 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:22.790640 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ace8d5-c13e-4dca-975b-acab59622990 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:22.808963 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-49ace8d5-c13e-4dca-975b-acab59622990 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:22.808963 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:22.808963 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:22.808963 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:22.808963 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:22.808963 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:22.808963 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:22.808963 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:22.808963 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:22.808963 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:22.808963 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:22.808963 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:22.808963 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:57:22.808963 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:57:22.808963 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:57:22.810105 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:57:22.810105 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:22.810105 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:22.810105 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:22.810105 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:22.810105 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:22.810105 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:22.810105 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:22.810105 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:57:22.810105 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:22.810105 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:22.810105 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:22.810105 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:22.810105 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:22.810105 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:22.810105 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:22.810105 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:22.811203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:57:22.811203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:57:22.811203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:22.811203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:57:22.811203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 626/1251] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:57:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:22.816040 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32888c18-92be-4758-acfa-80f0b3d8249b None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:22.932353 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32888c18-92be-4758-acfa-80f0b3d8249b None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:22.974023 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b76ed-817e-46b7-a31a-50bf536a0fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:22.974023 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b76ed-817e-46b7-a31a-50bf536a0fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:22.974023 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b76ed-817e-46b7-a31a-50bf536a0fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:22.974023 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b76ed-817e-46b7-a31a-50bf536a0fe8 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:22.974417 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b76ed-817e-46b7-a31a-50bf536a0fe8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:22.974511 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b76ed-817e-46b7-a31a-50bf536a0fe8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:22.975838 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b76ed-817e-46b7-a31a-50bf536a0fe8 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:22.989037 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 626/1252] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:22.995009 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ace8d5-c13e-4dca-975b-acab59622990 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:23.113062 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ace8d5-c13e-4dca-975b-acab59622990 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:23.140445 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:23.148904 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c6172-2b8d-4983-b57d-dc4ad37ec9a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:23.149723 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c6172-2b8d-4983-b57d-dc4ad37ec9a7 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:23.150009 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c6172-2b8d-4983-b57d-dc4ad37ec9a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:23.150279 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48c6172-2b8d-4983-b57d-dc4ad37ec9a7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:23.158282 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c6172-2b8d-4983-b57d-dc4ad37ec9a7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:23.159859 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48c6172-2b8d-4983-b57d-dc4ad37ec9a7 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:23.175281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-f48c6172-2b8d-4983-b57d-dc4ad37ec9a7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 16:57:23.175281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:23.175281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:23.175281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:23.175281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:23.175281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:23.175281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:23.175281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:23.175281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:23.175281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:23.175281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:23.175281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:23.175281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:57:23.175281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:57:23.175281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:57:23.175281 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:57:23.176795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:23.176795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:23.176795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:57:23.176795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:57:23.176795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:57:23.176795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:57:23.176795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:23.176795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:23.176795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:23.176795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:23.176795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:23.176795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:57:23.176795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:23.176795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:23.176795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:23.176795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:23.176795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:23.176795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:23.177968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:23.177968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:23.177968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:57:23.177968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:57:23.177968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:57:23.177968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:57:23.177968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:57:23.177968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:57:23.177968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 16:57:23.177968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:57:23.177968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 627/1253] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:57:22 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:57:23.183285 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b76ed-817e-46b7-a31a-50bf536a0fe8 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:23.243433 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:23.298215 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b76ed-817e-46b7-a31a-50bf536a0fe8 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:23.343158 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd58b1-a60e-403a-acbc-4a4429daf0eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:23.344017 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd58b1-a60e-403a-acbc-4a4429daf0eb None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:23.344378 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd58b1-a60e-403a-acbc-4a4429daf0eb None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:23.344855 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fd58b1-a60e-403a-acbc-4a4429daf0eb None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:23.346013 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd58b1-a60e-403a-acbc-4a4429daf0eb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:23.346486 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd58b1-a60e-403a-acbc-4a4429daf0eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:23.348941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fd58b1-a60e-403a-acbc-4a4429daf0eb None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:23.368617 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 627/1254] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 23 16:57:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:23.374618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c6172-2b8d-4983-b57d-dc4ad37ec9a7 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:23.490665 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48c6172-2b8d-4983-b57d-dc4ad37ec9a7 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:23.528805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ecc89-439f-4276-8f36-317f0c658056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:23.529386 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ecc89-439f-4276-8f36-317f0c658056 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:23.529674 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ecc89-439f-4276-8f36-317f0c658056 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:23.529945 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-148ecc89-439f-4276-8f36-317f0c658056 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:23.585713 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:23.601592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ecc89-439f-4276-8f36-317f0c658056 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:23.603184 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-148ecc89-439f-4276-8f36-317f0c658056 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:23.713003 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 628/1255] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:57:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 16:57:23.719566 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd58b1-a60e-403a-acbc-4a4429daf0eb None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:23.834454 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fd58b1-a60e-403a-acbc-4a4429daf0eb None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:23.850071 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:23.875585 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c318f-01f4-4b30-8a4a-36135160ed3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:23.876457 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c318f-01f4-4b30-8a4a-36135160ed3c None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:23.876804 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c318f-01f4-4b30-8a4a-36135160ed3c None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:23.877075 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489c318f-01f4-4b30-8a4a-36135160ed3c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:23.889700 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c318f-01f4-4b30-8a4a-36135160ed3c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:23.891234 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-489c318f-01f4-4b30-8a4a-36135160ed3c None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:23.922160 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-489c318f-01f4-4b30-8a4a-36135160ed3c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:23.922160 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:23.922160 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:23.922160 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:23.922160 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:23.922160 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:23.922160 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:23.922160 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:23.922160 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:23.922160 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:23.922160 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:23.922160 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:23.922160 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:57:23.922160 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:57:23.922160 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:57:23.923348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:57:23.923348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:23.923348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:23.923348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:23.923348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:23.923348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:23.923348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:23.923348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:23.923348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:57:23.923348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:23.923348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:23.923348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:23.923348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:23.923348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:23.923348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:23.923348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:23.923348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:23.924394 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:57:23.924394 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:57:23.924394 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:23.924394 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:57:23.924394 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 628/1256] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:57:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:23.930799 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ecc89-439f-4276-8f36-317f0c658056 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:24.030086 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:24.045796 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-148ecc89-439f-4276-8f36-317f0c658056 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:24.086883 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f3800-2745-42a6-8957-6ce32548dfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:24.087577 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f3800-2745-42a6-8957-6ce32548dfd4 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:24.088006 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f3800-2745-42a6-8957-6ce32548dfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:24.088322 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3f3800-2745-42a6-8957-6ce32548dfd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:24.089318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f3800-2745-42a6-8957-6ce32548dfd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:24.089645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f3800-2745-42a6-8957-6ce32548dfd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:24.091411 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca3f3800-2745-42a6-8957-6ce32548dfd4 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:24.103495 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 629/1257] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:57:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:24.109437 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c318f-01f4-4b30-8a4a-36135160ed3c None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:24.215767 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489c318f-01f4-4b30-8a4a-36135160ed3c None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:24.260650 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99346cf-6f1a-46f3-bea1-19830d906b02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:24.261856 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99346cf-6f1a-46f3-bea1-19830d906b02 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:24.262211 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99346cf-6f1a-46f3-bea1-19830d906b02 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:24.262517 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99346cf-6f1a-46f3-bea1-19830d906b02 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:24.271654 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99346cf-6f1a-46f3-bea1-19830d906b02 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:24.273395 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99346cf-6f1a-46f3-bea1-19830d906b02 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:24.290423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-a99346cf-6f1a-46f3-bea1-19830d906b02 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:57:24.290423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:24.290423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:24.290423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:24.290423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:24.290423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:24.290423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:24.290423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:24.290423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:24.290423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:24.290423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:24.290423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:24.290423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:57:24.290423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:57:24.290423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:57:24.291790 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:57:24.291790 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:24.291790 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:24.291790 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:24.291790 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:24.291790 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:24.291790 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:24.291790 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:24.291790 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:57:24.291790 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:24.291790 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:24.291790 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:24.291790 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:24.291790 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:24.291790 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:24.291790 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:24.291790 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:24.293238 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:57:24.293238 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:57:24.293238 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:57:24.293238 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:57:24.293238 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:57:24.293238 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:57:24.293238 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:57:24.293238 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:57:24.293238 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 629/1258] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:57:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:24.299786 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f3800-2745-42a6-8957-6ce32548dfd4 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:24.416232 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3f3800-2745-42a6-8957-6ce32548dfd4 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:24.456180 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882d1f97-d82f-40f1-a536-f900f5b0bfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:24.456989 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d1f97-d82f-40f1-a536-f900f5b0bfb2 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:24.457268 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d1f97-d82f-40f1-a536-f900f5b0bfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:24.457582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882d1f97-d82f-40f1-a536-f900f5b0bfb2 None admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:24.458559 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882d1f97-d82f-40f1-a536-f900f5b0bfb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:24.459017 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882d1f97-d82f-40f1-a536-f900f5b0bfb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:24.460826 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-882d1f97-d82f-40f1-a536-f900f5b0bfb2 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:24.477686 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 630/1259] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:57:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:24.484541 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99346cf-6f1a-46f3-bea1-19830d906b02 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:24.514526 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:24.606275 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99346cf-6f1a-46f3-bea1-19830d906b02 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:24.654892 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d731f84f-58eb-470b-8cd2-ef80607ff4bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:24.655585 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731f84f-58eb-470b-8cd2-ef80607ff4bc None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:24.655871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731f84f-58eb-470b-8cd2-ef80607ff4bc None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:24.656147 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d731f84f-58eb-470b-8cd2-ef80607ff4bc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:24.656976 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731f84f-58eb-470b-8cd2-ef80607ff4bc None admin] RBAC: Adding query filter params (role.id=17bbe7cc1c3641509de5e9dca8534102, scope.project.id=0595667fb2e04a3ca4716f669d6698b0, user.id=f181ebdfcd544b54af7e6f9c118fe252) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:24.657292 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731f84f-58eb-470b-8cd2-ef80607ff4bc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:24.658731 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d731f84f-58eb-470b-8cd2-ef80607ff4bc None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:24.665802 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 630/1260] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:57:24 2022] GET /identity/v3/role_assignments?user.id=f181ebdfcd544b54af7e6f9c118fe252&scope.project.id=0595667fb2e04a3ca4716f669d6698b0&role.id=17bbe7cc1c3641509de5e9dca8534102 => generated 259 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:26.832673 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c5cf9-8f06-41b1-976b-ba2ba7978815 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:26.833502 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c5cf9-8f06-41b1-976b-ba2ba7978815 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:26.833502 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378c5cf9-8f06-41b1-976b-ba2ba7978815 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:26.835590 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 631/1261] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:57:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:57:26.842151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622e47c-10b6-4496-9f13-44fcb432254e None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:26.842608 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622e47c-10b6-4496-9f13-44fcb432254e None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:26.842699 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6622e47c-10b6-4496-9f13-44fcb432254e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:26.870998 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-6622e47c-10b6-4496-9f13-44fcb432254e None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:57:26.872194 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-6622e47c-10b6-4496-9f13-44fcb432254e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:57:26.944872 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 631/1262] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:57:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:57:26.974642 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c5cf9-8f06-41b1-976b-ba2ba7978815 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:26.987156 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:27.093756 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378c5cf9-8f06-41b1-976b-ba2ba7978815 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:27.132265 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b32ff2-c0c4-4535-99f7-2b8e8d04e9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:27.133051 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b32ff2-c0c4-4535-99f7-2b8e8d04e9cf None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:27.133329 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b32ff2-c0c4-4535-99f7-2b8e8d04e9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:27.133603 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b32ff2-c0c4-4535-99f7-2b8e8d04e9cf None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:27.145164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b32ff2-c0c4-4535-99f7-2b8e8d04e9cf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:57:27.152909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-62b32ff2-c0c4-4535-99f7-2b8e8d04e9cf None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:57:27.152909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:27.152909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:27.152909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:27.152909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:27.152909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:27.152909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:27.152909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:27.152909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:27.152909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:27.152909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:27.152909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:27.152909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:57:27.152909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:57:27.152909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:57:27.152909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:57:27.154060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:57:27.154060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:57:27.154060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:57:27.154060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:57:27.154060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:27.154060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:27.154060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:27.154060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:27.154060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:27.154060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:27.154060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:27.154060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:57:27.154060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:27.154060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:27.154060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:27.154060 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:27.154950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:27.154950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:27.154950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:27.154950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:27.154950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:57:27.154950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:57:27.154950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:57:27.154950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:57:27.154950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:57:27.154950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:57:27.154950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:57:27.154950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:57:27.154950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 632/1263] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:57:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:27.162726 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622e47c-10b6-4496-9f13-44fcb432254e None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:27.183742 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:27.268471 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6622e47c-10b6-4496-9f13-44fcb432254e None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:27.315604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb094a46-35dd-4b48-ba09-abba4dfcea6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:27.316391 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb094a46-35dd-4b48-ba09-abba4dfcea6c None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:27.316668 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb094a46-35dd-4b48-ba09-abba4dfcea6c None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:27.316987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb094a46-35dd-4b48-ba09-abba4dfcea6c None admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:27.317797 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb094a46-35dd-4b48-ba09-abba4dfcea6c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:27.318103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb094a46-35dd-4b48-ba09-abba4dfcea6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:27.319519 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb094a46-35dd-4b48-ba09-abba4dfcea6c None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:27.330137 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 632/1264] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:57:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:27.336823 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b32ff2-c0c4-4535-99f7-2b8e8d04e9cf None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:27.405914 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:27.439968 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b32ff2-c0c4-4535-99f7-2b8e8d04e9cf None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:27.476129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32630dad-079e-404e-80cd-21ce77e3c37f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:27.476906 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32630dad-079e-404e-80cd-21ce77e3c37f None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:27.477179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32630dad-079e-404e-80cd-21ce77e3c37f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:27.477447 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32630dad-079e-404e-80cd-21ce77e3c37f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:27.489887 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32630dad-079e-404e-80cd-21ce77e3c37f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:27.491365 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32630dad-079e-404e-80cd-21ce77e3c37f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:27.508879 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-32630dad-079e-404e-80cd-21ce77e3c37f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:27.508879 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:27.508879 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:27.508879 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:27.508879 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:27.508879 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:27.508879 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:27.508879 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:27.508879 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:27.508879 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:27.508879 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:27.508879 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:27.508879 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:57:27.508879 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:57:27.508879 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:57:27.509775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:57:27.509775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:27.509775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:27.509775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:27.509775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:27.509775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:27.509775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:27.509775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:27.509775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:57:27.509775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:27.509775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:27.509775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:27.509775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:27.509775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:27.509775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:27.509775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:27.509775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:27.511229 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:57:27.511229 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:57:27.511229 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:27.511229 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:57:27.511229 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 633/1265] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:57:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:27.516009 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb094a46-35dd-4b48-ba09-abba4dfcea6c None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:27.612444 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:27.634244 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb094a46-35dd-4b48-ba09-abba4dfcea6c None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:27.670655 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61641b51-3ba8-46c1-bd65-43f68f0932f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:27.671271 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61641b51-3ba8-46c1-bd65-43f68f0932f7 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:27.671549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61641b51-3ba8-46c1-bd65-43f68f0932f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:27.671820 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61641b51-3ba8-46c1-bd65-43f68f0932f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:27.672531 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61641b51-3ba8-46c1-bd65-43f68f0932f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:27.672839 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61641b51-3ba8-46c1-bd65-43f68f0932f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:27.674185 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61641b51-3ba8-46c1-bd65-43f68f0932f7 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:27.687172 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 633/1266] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:57:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:27.696342 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32630dad-079e-404e-80cd-21ce77e3c37f None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:27.808587 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32630dad-079e-404e-80cd-21ce77e3c37f None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:27.828484 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:27.845926 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57f0a4-a127-4330-bd15-22186bb678df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:27.846720 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57f0a4-a127-4330-bd15-22186bb678df None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:27.847026 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57f0a4-a127-4330-bd15-22186bb678df None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:27.847299 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd57f0a4-a127-4330-bd15-22186bb678df None admin] PATH_INFO: `/v3/users/placement` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:27.856140 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57f0a4-a127-4330-bd15-22186bb678df None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:27.857620 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd57f0a4-a127-4330-bd15-22186bb678df None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:27.871899 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-fd57f0a4-a127-4330-bd15-22186bb678df None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 16:57:27.871899 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:27.871899 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:27.871899 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:27.871899 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:27.871899 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:27.871899 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:27.871899 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:27.871899 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:27.871899 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:27.871899 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:27.871899 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:27.871899 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:57:27.871899 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:57:27.871899 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:57:27.871899 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:57:27.872966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:27.872966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:27.872966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:57:27.872966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:57:27.872966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:57:27.872966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:57:27.872966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:27.872966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:27.872966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:27.872966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:27.872966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:27.872966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:57:27.872966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:27.872966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:27.872966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:27.872966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:27.872966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:27.872966 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:27.874292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:27.874292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:27.874292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:57:27.874292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:57:27.874292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:57:27.874292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:57:27.874292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:57:27.874292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:57:27.874292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 16:57:27.874292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:57:27.874292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 634/1267] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:57:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:27.879278 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61641b51-3ba8-46c1-bd65-43f68f0932f7 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:27.981099 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61641b51-3ba8-46c1-bd65-43f68f0932f7 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:28.018796 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff805ac7-13c7-42c2-96d6-a7dddcf5ee62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:28.019485 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff805ac7-13c7-42c2-96d6-a7dddcf5ee62 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:28.019763 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff805ac7-13c7-42c2-96d6-a7dddcf5ee62 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:28.020032 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff805ac7-13c7-42c2-96d6-a7dddcf5ee62 None admin] PATH_INFO: `/v3/users` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:28.020866 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff805ac7-13c7-42c2-96d6-a7dddcf5ee62 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:28.021171 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff805ac7-13c7-42c2-96d6-a7dddcf5ee62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:28.022595 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff805ac7-13c7-42c2-96d6-a7dddcf5ee62 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:28.031379 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:28.044210 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 634/1268] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 23 16:57:27 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:28.049988 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57f0a4-a127-4330-bd15-22186bb678df None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:28.153100 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd57f0a4-a127-4330-bd15-22186bb678df None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:28.188094 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110820a-928e-413d-b68a-250191222d45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:28.188865 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110820a-928e-413d-b68a-250191222d45 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:28.189146 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110820a-928e-413d-b68a-250191222d45 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:28.189420 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f110820a-928e-413d-b68a-250191222d45 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:28.200361 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110820a-928e-413d-b68a-250191222d45 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:28.201839 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f110820a-928e-413d-b68a-250191222d45 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:28.219021 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-f110820a-928e-413d-b68a-250191222d45 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:28.219021 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:28.219021 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:28.219021 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:28.219021 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:28.219021 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:28.219021 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:28.219021 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:28.219021 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:28.219021 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:28.219021 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:28.219021 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:28.219021 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:57:28.219021 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:57:28.219021 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:57:28.220011 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:57:28.220011 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:28.220011 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:28.220011 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:28.220011 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:28.220011 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:28.220011 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:28.220011 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:28.220011 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:57:28.220011 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:28.220011 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:28.220011 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:28.220011 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:28.220011 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:28.220011 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:28.220011 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:28.220011 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:28.221264 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:57:28.221264 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:57:28.221264 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:28.221264 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:57:28.221264 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 635/1269] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:57:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:28.229637 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff805ac7-13c7-42c2-96d6-a7dddcf5ee62 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:28.356740 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff805ac7-13c7-42c2-96d6-a7dddcf5ee62 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:28.399263 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df136f0-bee9-41dd-a33c-4965539dbb26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:28.400237 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df136f0-bee9-41dd-a33c-4965539dbb26 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:28.400607 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df136f0-bee9-41dd-a33c-4965539dbb26 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:28.401535 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df136f0-bee9-41dd-a33c-4965539dbb26 None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:28.402374 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df136f0-bee9-41dd-a33c-4965539dbb26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:28.402685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df136f0-bee9-41dd-a33c-4965539dbb26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:28.404169 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df136f0-bee9-41dd-a33c-4965539dbb26 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:28.417721 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 635/1270] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:57:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:28.423686 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110820a-928e-413d-b68a-250191222d45 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:28.457938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:28.534789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f110820a-928e-413d-b68a-250191222d45 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:28.570263 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc092408-3c1b-446e-887a-00c21ddcdf58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:28.571078 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc092408-3c1b-446e-887a-00c21ddcdf58 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:28.571359 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc092408-3c1b-446e-887a-00c21ddcdf58 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:28.571633 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc092408-3c1b-446e-887a-00c21ddcdf58 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:28.578898 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc092408-3c1b-446e-887a-00c21ddcdf58 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:28.580375 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc092408-3c1b-446e-887a-00c21ddcdf58 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:28.594825 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-fc092408-3c1b-446e-887a-00c21ddcdf58 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:57:28.594825 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:28.594825 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:28.594825 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:28.594825 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:28.594825 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:28.594825 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:28.594825 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:28.594825 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:28.594825 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:28.594825 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:28.594825 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:28.594825 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:57:28.594825 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:57:28.594825 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:57:28.595871 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:57:28.595871 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:28.595871 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:28.595871 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:28.595871 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:28.595871 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:28.595871 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:28.595871 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:28.595871 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:57:28.595871 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:28.595871 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:28.595871 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:28.595871 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:28.595871 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:28.595871 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:28.595871 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:28.595871 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:28.596822 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:57:28.596822 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:57:28.596822 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:57:28.596822 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:57:28.596822 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:57:28.596822 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:57:28.596822 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:57:28.596822 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:57:28.596822 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 636/1271] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:57:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:28.604118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df136f0-bee9-41dd-a33c-4965539dbb26 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:28.664985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:28.715822 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df136f0-bee9-41dd-a33c-4965539dbb26 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:28.760626 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d17678-1298-4918-9260-edc8c6772b79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:28.761491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d17678-1298-4918-9260-edc8c6772b79 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:28.761770 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d17678-1298-4918-9260-edc8c6772b79 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:28.762042 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d17678-1298-4918-9260-edc8c6772b79 None admin] PATH_INFO: `/v3/projects` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:28.762824 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d17678-1298-4918-9260-edc8c6772b79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:28.763164 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d17678-1298-4918-9260-edc8c6772b79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:28.764586 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d17678-1298-4918-9260-edc8c6772b79 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:28.776599 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 636/1272] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:57:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:28.783759 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc092408-3c1b-446e-887a-00c21ddcdf58 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:28.894451 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:28.904687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc092408-3c1b-446e-887a-00c21ddcdf58 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:28.943433 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ddc1dd-ec5b-410f-933a-8ff6443e8124 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:28.944267 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddc1dd-ec5b-410f-933a-8ff6443e8124 None admin] REQUEST_METHOD: `PUT` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:28.944521 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddc1dd-ec5b-410f-933a-8ff6443e8124 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:28.944789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ddc1dd-ec5b-410f-933a-8ff6443e8124 None admin] PATH_INFO: `/v3/projects/0595667fb2e04a3ca4716f669d6698b0/users/f181ebdfcd544b54af7e6f9c118fe252/roles/17bbe7cc1c3641509de5e9dca8534102` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:28.946801 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ddc1dd-ec5b-410f-933a-8ff6443e8124 None admin] RBAC: Authorizing `identity:create_grant(project_id=0595667fb2e04a3ca4716f669d6698b0, user_id=f181ebdfcd544b54af7e6f9c118fe252, role_id=17bbe7cc1c3641509de5e9dca8534102)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:28.948517 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ddc1dd-ec5b-410f-933a-8ff6443e8124 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:28.966487 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 637/1273] 10.222.0.13 () {58 vars in 1439 bytes} [Mon May 23 16:57:28 2022] PUT /identity/v3/projects/0595667fb2e04a3ca4716f669d6698b0/users/f181ebdfcd544b54af7e6f9c118fe252/roles/17bbe7cc1c3641509de5e9dca8534102 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 16:57:31.158822 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac9f00-fba3-4332-b614-f39103d025c0 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:31.159765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac9f00-fba3-4332-b614-f39103d025c0 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:31.159765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ac9f00-fba3-4332-b614-f39103d025c0 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:31.161920 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 637/1274] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:57:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:57:31.168645 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e4d24-a131-47e3-9a15-c1022ed6146b None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:31.169346 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e4d24-a131-47e3-9a15-c1022ed6146b None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:31.169495 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84e4d24-a131-47e3-9a15-c1022ed6146b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:31.197405 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-e84e4d24-a131-47e3-9a15-c1022ed6146b None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:57:31.198647 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e84e4d24-a131-47e3-9a15-c1022ed6146b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:57:31.270311 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 638/1275] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:57:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:57:31.297573 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac9f00-fba3-4332-b614-f39103d025c0 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:31.419580 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ac9f00-fba3-4332-b614-f39103d025c0 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:31.450036 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:31.468912 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d32527-832f-45d4-b94a-2cbc77233526 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:31.469714 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d32527-832f-45d4-b94a-2cbc77233526 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:31.469989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d32527-832f-45d4-b94a-2cbc77233526 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:31.470265 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d32527-832f-45d4-b94a-2cbc77233526 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:31.482738 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d32527-832f-45d4-b94a-2cbc77233526 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 16:57:31.490261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-b9d32527-832f-45d4-b94a-2cbc77233526 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:57:31.490261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:31.490261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:31.490261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:31.490261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:31.490261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:31.490261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:31.490261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:31.490261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:31.490261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:31.490261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:31.490261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:31.490261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 16:57:31.490261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 16:57:31.490261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 16:57:31.490261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 16:57:31.491281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 16:57:31.491281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 16:57:31.491281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 16:57:31.491281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 16:57:31.491281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:31.491281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:31.491281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:31.491281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:31.491281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:31.491281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:31.491281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:31.491281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:57:31.491281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:31.491281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:31.491281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:31.491281 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:31.492278 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:31.492278 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:31.492278 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:31.492278 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:31.492278 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 16:57:31.492278 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 16:57:31.492278 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 16:57:31.492278 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 16:57:31.492278 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 16:57:31.492278 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 16:57:31.492278 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 16:57:31.492278 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:57:31.492278 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 638/1276] 10.222.0.13 () {56 vars in 1096 bytes} [Mon May 23 16:57:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:31.497943 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e4d24-a131-47e3-9a15-c1022ed6146b None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:31.580901 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:31.608936 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84e4d24-a131-47e3-9a15-c1022ed6146b None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:31.648326 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd360188-e1ed-401b-8975-6434fb72f087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:31.649360 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd360188-e1ed-401b-8975-6434fb72f087 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:31.649830 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd360188-e1ed-401b-8975-6434fb72f087 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:31.650244 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd360188-e1ed-401b-8975-6434fb72f087 None admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:31.651517 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd360188-e1ed-401b-8975-6434fb72f087 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:31.652022 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd360188-e1ed-401b-8975-6434fb72f087 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:31.654229 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd360188-e1ed-401b-8975-6434fb72f087 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:31.667507 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 639/1277] 10.222.0.13 () {56 vars in 1099 bytes} [Mon May 23 16:57:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:31.673602 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d32527-832f-45d4-b94a-2cbc77233526 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:31.780135 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d32527-832f-45d4-b94a-2cbc77233526 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:31.816614 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d61eef-1c39-41ea-844e-a782e4126152 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:31.817412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d61eef-1c39-41ea-844e-a782e4126152 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:31.817688 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d61eef-1c39-41ea-844e-a782e4126152 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:31.817919 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d61eef-1c39-41ea-844e-a782e4126152 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:31.829031 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d61eef-1c39-41ea-844e-a782e4126152 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:31.830488 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d61eef-1c39-41ea-844e-a782e4126152 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:31.847932 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-68d61eef-1c39-41ea-844e-a782e4126152 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:31.847932 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:31.847932 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:31.847932 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:31.847932 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:31.847932 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:31.847932 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:31.847932 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:31.847932 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:31.847932 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:31.847932 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:31.847932 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:31.847932 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:57:31.847932 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:57:31.847932 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:57:31.849179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:57:31.849179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:31.849179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:31.849179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:31.849179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:31.849179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:31.849179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:31.849179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:31.849179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:57:31.849179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:31.849179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:31.849179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:31.849179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:31.849179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:31.849179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:31.849179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:31.849179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:31.850261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:57:31.850261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:57:31.850261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:31.850261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:57:31.850261 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 639/1278] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:57:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:31.855681 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd360188-e1ed-401b-8975-6434fb72f087 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:31.981203 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd360188-e1ed-401b-8975-6434fb72f087 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:32.017429 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2566d5-d8e1-43ea-96e2-ac587e54be72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:32.018097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2566d5-d8e1-43ea-96e2-ac587e54be72 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:32.018377 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2566d5-d8e1-43ea-96e2-ac587e54be72 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:32.018643 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2566d5-d8e1-43ea-96e2-ac587e54be72 None admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:32.019369 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2566d5-d8e1-43ea-96e2-ac587e54be72 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:32.019673 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2566d5-d8e1-43ea-96e2-ac587e54be72 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:32.021045 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2566d5-d8e1-43ea-96e2-ac587e54be72 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:32.037731 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 640/1279] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:57:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:32.038966 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d61eef-1c39-41ea-844e-a782e4126152 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:32.071443 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:32.141765 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d61eef-1c39-41ea-844e-a782e4126152 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:32.174630 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9c833-6dea-4e16-a597-8d02080aac9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:32.175463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9c833-6dea-4e16-a597-8d02080aac9a None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:32.175740 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9c833-6dea-4e16-a597-8d02080aac9a None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:32.176012 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce9c833-6dea-4e16-a597-8d02080aac9a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:32.183728 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9c833-6dea-4e16-a597-8d02080aac9a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:32.185177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce9c833-6dea-4e16-a597-8d02080aac9a None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:32.199518 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-2ce9c833-6dea-4e16-a597-8d02080aac9a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 16:57:32.199518 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:32.199518 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:32.199518 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:32.199518 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:32.199518 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:32.199518 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:32.199518 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:32.199518 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:32.199518 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:32.199518 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:32.199518 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:32.199518 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 16:57:32.199518 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 16:57:32.199518 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 16:57:32.199518 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 16:57:32.200477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:32.200477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:32.200477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 16:57:32.200477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:57:32.200477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 16:57:32.200477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 16:57:32.200477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:32.200477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:32.200477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:32.200477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:32.200477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:32.200477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:57:32.200477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:32.200477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:32.200477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:32.200477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:32.200477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:32.200477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:32.201884 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:32.201884 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:32.201884 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 16:57:32.201884 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 16:57:32.201884 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 16:57:32.201884 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 16:57:32.201884 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 16:57:32.201884 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 16:57:32.201884 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 16:57:32.201884 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:57:32.201884 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 640/1280] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:57:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:32.206569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2566d5-d8e1-43ea-96e2-ac587e54be72 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:32.210585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:32.307206 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2566d5-d8e1-43ea-96e2-ac587e54be72 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:32.341823 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215ff6f-f25e-43df-b599-241730a18e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:32.343230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215ff6f-f25e-43df-b599-241730a18e68 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:32.343733 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215ff6f-f25e-43df-b599-241730a18e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:32.344006 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2215ff6f-f25e-43df-b599-241730a18e68 None admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:32.344834 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2215ff6f-f25e-43df-b599-241730a18e68 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:32.345141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2215ff6f-f25e-43df-b599-241730a18e68 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:32.346567 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2215ff6f-f25e-43df-b599-241730a18e68 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:32.363762 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 641/1281] 10.222.0.13 () {56 vars in 1143 bytes} [Mon May 23 16:57:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:32.369872 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9c833-6dea-4e16-a597-8d02080aac9a None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:32.471126 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:32.482256 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce9c833-6dea-4e16-a597-8d02080aac9a None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:32.520115 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512003f0-fa3a-4e6c-8541-8b06fb9e7dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:32.520755 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512003f0-fa3a-4e6c-8541-8b06fb9e7dc6 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:32.521035 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512003f0-fa3a-4e6c-8541-8b06fb9e7dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:32.521301 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512003f0-fa3a-4e6c-8541-8b06fb9e7dc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:32.590172 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512003f0-fa3a-4e6c-8541-8b06fb9e7dc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:32.591641 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512003f0-fa3a-4e6c-8541-8b06fb9e7dc6 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:32.699148 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:32.710185 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 641/1282] 10.222.0.13 () {58 vars in 1282 bytes} [Mon May 23 16:57:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 16:57:32.717438 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215ff6f-f25e-43df-b599-241730a18e68 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:32.779581 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:32.818316 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2215ff6f-f25e-43df-b599-241730a18e68 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:32.852736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e74b5-77ec-4393-b5de-b0ba4547e627 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:32.853454 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e74b5-77ec-4393-b5de-b0ba4547e627 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:32.853761 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e74b5-77ec-4393-b5de-b0ba4547e627 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:32.854072 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33e74b5-77ec-4393-b5de-b0ba4547e627 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:32.864234 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e74b5-77ec-4393-b5de-b0ba4547e627 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:32.865679 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33e74b5-77ec-4393-b5de-b0ba4547e627 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:32.883542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-b33e74b5-77ec-4393-b5de-b0ba4547e627 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:32.883542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:32.883542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:32.883542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:32.883542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:32.883542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:32.883542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:32.883542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:32.883542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:32.883542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:32.883542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:32.883542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:32.883542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 16:57:32.883542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 16:57:32.883542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 16:57:32.884569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 16:57:32.884569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:32.884569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:32.884569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:32.884569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:32.884569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:32.884569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:32.884569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:32.884569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:57:32.884569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:32.884569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:32.884569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:32.884569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:32.884569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:32.884569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:32.884569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:32.884569 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:32.885637 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 16:57:32.885637 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 16:57:32.885637 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 16:57:32.885637 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:57:32.887105 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 642/1283] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 16:57:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:32.890511 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512003f0-fa3a-4e6c-8541-8b06fb9e7dc6 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:32.998185 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512003f0-fa3a-4e6c-8541-8b06fb9e7dc6 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:33.037644 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8da15-ea7b-4f31-a040-de051965454b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:33.038308 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8da15-ea7b-4f31-a040-de051965454b None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:33.038593 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8da15-ea7b-4f31-a040-de051965454b None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:33.038871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8da15-ea7b-4f31-a040-de051965454b None admin] PATH_INFO: `/v3/domains` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:33.039629 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8da15-ea7b-4f31-a040-de051965454b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:33.039937 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8da15-ea7b-4f31-a040-de051965454b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:33.041359 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8da15-ea7b-4f31-a040-de051965454b None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:33.052591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 642/1284] 10.222.0.13 () {56 vars in 1109 bytes} [Mon May 23 16:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:33.059440 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e74b5-77ec-4393-b5de-b0ba4547e627 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:33.173011 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33e74b5-77ec-4393-b5de-b0ba4547e627 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:33.187834 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:33.212478 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bec70b-05d7-4f65-b32b-1269e8eb3352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:33.213377 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bec70b-05d7-4f65-b32b-1269e8eb3352 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:33.213660 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bec70b-05d7-4f65-b32b-1269e8eb3352 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:33.213936 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5bec70b-05d7-4f65-b32b-1269e8eb3352 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:33.220622 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bec70b-05d7-4f65-b32b-1269e8eb3352 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:33.222076 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5bec70b-05d7-4f65-b32b-1269e8eb3352 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:33.235616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application [None req-e5bec70b-05d7-4f65-b32b-1269e8eb3352 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:57:33.235616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:33.235616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:33.235616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:33.235616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:33.235616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:33.235616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:33.235616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:33.235616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:33.235616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:33.235616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:33.235616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:33.235616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 16:57:33.235616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 16:57:33.235616 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 16:57:33.236619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 16:57:33.236619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:33.236619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:33.236619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:33.236619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:33.236619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:33.236619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:33.236619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:33.236619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application with Lock( May 23 16:57:33.236619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:33.236619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:33.236619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:33.236619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:33.236619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:33.236619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:33.236619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:33.236619 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:33.237613 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 16:57:33.237613 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 16:57:33.237613 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 16:57:33.237613 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 16:57:33.237613 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 16:57:33.237613 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 16:57:33.237613 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 16:57:33.237613 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: ERROR keystone.server.flask.application May 23 16:57:33.237613 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 643/1285] 10.222.0.13 () {56 vars in 1111 bytes} [Mon May 23 16:57:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:33.243126 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8da15-ea7b-4f31-a040-de051965454b None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:33.296590 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:33.347821 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8da15-ea7b-4f31-a040-de051965454b None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:33.383225 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e792f941-3341-4761-9c83-96a5907e88e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:33.383882 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792f941-3341-4761-9c83-96a5907e88e2 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:33.384170 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792f941-3341-4761-9c83-96a5907e88e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:33.384445 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e792f941-3341-4761-9c83-96a5907e88e2 None admin] PATH_INFO: `/v3/projects` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:33.385306 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792f941-3341-4761-9c83-96a5907e88e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:33.385591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792f941-3341-4761-9c83-96a5907e88e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:33.387050 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e792f941-3341-4761-9c83-96a5907e88e2 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:33.400221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 643/1286] 10.222.0.13 () {56 vars in 1148 bytes} [Mon May 23 16:57:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:33.407665 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bec70b-05d7-4f65-b32b-1269e8eb3352 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:33.517907 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5bec70b-05d7-4f65-b32b-1269e8eb3352 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:33.552071 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e7f131-0d7e-46db-8636-49ed6166e7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:33.552707 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7f131-0d7e-46db-8636-49ed6166e7ca None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:33.552982 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7f131-0d7e-46db-8636-49ed6166e7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:33.553245 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e7f131-0d7e-46db-8636-49ed6166e7ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:33.554072 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7f131-0d7e-46db-8636-49ed6166e7ca None admin] RBAC: Adding query filter params (role.id=17bbe7cc1c3641509de5e9dca8534102, scope.project.id=0595667fb2e04a3ca4716f669d6698b0, user.id=f181ebdfcd544b54af7e6f9c118fe252) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:33.554385 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7f131-0d7e-46db-8636-49ed6166e7ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:33.555838 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e7f131-0d7e-46db-8636-49ed6166e7ca None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:33.561905 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 644/1287] 10.222.0.13 () {56 vars in 1374 bytes} [Mon May 23 16:57:33 2022] GET /identity/v3/role_assignments?user.id=f181ebdfcd544b54af7e6f9c118fe252&scope.project.id=0595667fb2e04a3ca4716f669d6698b0&role.id=17bbe7cc1c3641509de5e9dca8534102 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:35.742280 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9743af-1379-47fc-962e-a493a687156b None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:35.743128 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9743af-1379-47fc-962e-a493a687156b None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:35.743128 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9743af-1379-47fc-962e-a493a687156b None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:35.744936 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 644/1288] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:57:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:57:35.751454 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8087e01-426b-4d10-932b-2e4e8524e97d None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:35.751898 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8087e01-426b-4d10-932b-2e4e8524e97d None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:35.752024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8087e01-426b-4d10-932b-2e4e8524e97d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:35.779128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-c8087e01-426b-4d10-932b-2e4e8524e97d None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:57:35.780314 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c8087e01-426b-4d10-932b-2e4e8524e97d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:57:35.811298 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:35.853345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 645/1289] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:57:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:57:35.876515 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9743af-1379-47fc-962e-a493a687156b None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:35.987973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9743af-1379-47fc-962e-a493a687156b None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:35.998626 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:36.027639 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640e1c9-9175-42c9-bfd4-6489389aac2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:36.028435 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640e1c9-9175-42c9-bfd4-6489389aac2f None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:36.028709 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640e1c9-9175-42c9-bfd4-6489389aac2f None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:36.028980 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640e1c9-9175-42c9-bfd4-6489389aac2f None admin] PATH_INFO: `/v3/services/placement` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:36.029621 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b640e1c9-9175-42c9-bfd4-6489389aac2f None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:36.031025 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b640e1c9-9175-42c9-bfd4-6489389aac2f None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:36.043242 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-b640e1c9-9175-42c9-bfd4-6489389aac2f None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 23 16:57:36.043242 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:36.043242 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:57:36.043242 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:57:36.043242 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:57:36.043242 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:57:36.043242 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:57:36.043242 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:57:36.043242 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 16:57:36.043242 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:57:36.043242 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:57:36.043242 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:57:36.043242 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:36.043242 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:36.043242 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:36.043242 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:36.043242 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:36.044408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:36.044408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:36.044408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:36.044408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:36.044408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:36.044408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:36.044408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:57:36.044408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:57:36.044408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:57:36.044408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:57:36.044408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:36.044408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:36.044408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:36.044408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:36.044408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:36.044408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:36.045436 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:36.045436 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:57:36.045436 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:36.045436 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:36.045436 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:36.045436 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:36.045436 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:36.045436 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:36.045436 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:36.045436 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:36.045436 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:57:36.045436 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:57:36.045436 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 16:57:36.045436 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:57:36.045436 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 645/1290] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 23 16:57:35 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:36.050791 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8087e01-426b-4d10-932b-2e4e8524e97d None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:36.158106 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8087e01-426b-4d10-932b-2e4e8524e97d None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:36.194193 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4807e8f-3a20-46d8-aa09-43c93fb94807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:36.194844 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4807e8f-3a20-46d8-aa09-43c93fb94807 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:36.195156 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4807e8f-3a20-46d8-aa09-43c93fb94807 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:36.195432 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4807e8f-3a20-46d8-aa09-43c93fb94807 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:36.196132 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4807e8f-3a20-46d8-aa09-43c93fb94807 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:36.196439 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4807e8f-3a20-46d8-aa09-43c93fb94807 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:36.197880 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4807e8f-3a20-46d8-aa09-43c93fb94807 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:36.202818 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 646/1291] 10.222.0.13 () {56 vars in 1116 bytes} [Mon May 23 16:57:36 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:36.209151 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640e1c9-9175-42c9-bfd4-6489389aac2f None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:36.327331 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640e1c9-9175-42c9-bfd4-6489389aac2f None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:36.368610 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c5d407-17da-412f-b1a9-54d4fa3d5b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:36.369628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5d407-17da-412f-b1a9-54d4fa3d5b0d None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:36.369628 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5d407-17da-412f-b1a9-54d4fa3d5b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:36.369952 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c5d407-17da-412f-b1a9-54d4fa3d5b0d None admin] PATH_INFO: `/v3/services` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:36.370662 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5d407-17da-412f-b1a9-54d4fa3d5b0d None admin] RBAC: Adding query filter params (type=placement) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:36.370981 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5d407-17da-412f-b1a9-54d4fa3d5b0d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:36.372395 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c5d407-17da-412f-b1a9-54d4fa3d5b0d None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:36.377329 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 646/1292] 10.222.0.13 () {56 vars in 1116 bytes} [Mon May 23 16:57:36 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:38.584311 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556084e-c0f5-4d44-a14f-75b6f02dcfdc None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:38.585140 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556084e-c0f5-4d44-a14f-75b6f02dcfdc None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:38.585140 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556084e-c0f5-4d44-a14f-75b6f02dcfdc None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:38.587401 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 647/1293] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:57:38 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:57:38.596885 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08c867-d142-4a31-b86f-4a4207037e97 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:38.597292 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08c867-d142-4a31-b86f-4a4207037e97 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:38.597503 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08c867-d142-4a31-b86f-4a4207037e97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:38.617417 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-0c08c867-d142-4a31-b86f-4a4207037e97 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:57:38.618600 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-0c08c867-d142-4a31-b86f-4a4207037e97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:57:38.649568 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:38.693482 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 647/1294] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:57:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:57:38.717419 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a556084e-c0f5-4d44-a14f-75b6f02dcfdc None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:38.762264 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:38.841896 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a556084e-c0f5-4d44-a14f-75b6f02dcfdc None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:38.876788 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14b654f-5744-4510-86f3-6f862204829f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:38.877437 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b654f-5744-4510-86f3-6f862204829f None admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:38.877710 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b654f-5744-4510-86f3-6f862204829f None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:38.878298 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14b654f-5744-4510-86f3-6f862204829f None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:38.879041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b654f-5744-4510-86f3-6f862204829f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:38.880407 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14b654f-5744-4510-86f3-6f862204829f None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:38.896127 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 648/1295] 10.222.0.13 () {60 vars in 1141 bytes} [Mon May 23 16:57:38 2022] POST /identity/v3/services => generated 246 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:57:41.174007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af5b6e-230b-4e7d-b40d-32ebcc2f8144 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:41.174596 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af5b6e-230b-4e7d-b40d-32ebcc2f8144 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:41.174866 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af5b6e-230b-4e7d-b40d-32ebcc2f8144 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:41.177161 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 648/1296] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:57:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:57:41.185261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb56b0-8205-4b14-a57e-88a017b9a3cd None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:41.185946 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb56b0-8205-4b14-a57e-88a017b9a3cd None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:41.186089 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4eb56b0-8205-4b14-a57e-88a017b9a3cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:41.214515 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-f4eb56b0-8205-4b14-a57e-88a017b9a3cd None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:57:41.215736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-f4eb56b0-8205-4b14-a57e-88a017b9a3cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:57:41.329464 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 649/1297] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:57:41 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:57:41.359650 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af5b6e-230b-4e7d-b40d-32ebcc2f8144 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:41.476469 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af5b6e-230b-4e7d-b40d-32ebcc2f8144 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:41.515933 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda2722-3d4b-4f31-8668-a63abe1002e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:41.516737 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda2722-3d4b-4f31-8668-a63abe1002e5 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:41.517022 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda2722-3d4b-4f31-8668-a63abe1002e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:41.517295 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bda2722-3d4b-4f31-8668-a63abe1002e5 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:41.517953 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda2722-3d4b-4f31-8668-a63abe1002e5 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:41.519375 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bda2722-3d4b-4f31-8668-a63abe1002e5 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:41.532506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-2bda2722-3d4b-4f31-8668-a63abe1002e5 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 23 16:57:41.532506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:41.532506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:57:41.532506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:57:41.532506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:57:41.532506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:57:41.532506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:57:41.532506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:57:41.532506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 16:57:41.532506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:57:41.532506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:57:41.532506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:57:41.532506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:41.532506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:41.532506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:41.532506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:41.532506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:41.533642 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:41.533642 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:41.533642 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:41.533642 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:41.533642 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:41.533642 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:41.533642 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:57:41.533642 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:57:41.533642 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:57:41.533642 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:57:41.533642 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:41.533642 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:41.533642 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:41.533642 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:41.533642 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:41.533642 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:41.535013 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:41.535013 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:57:41.535013 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:41.535013 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:41.535013 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:41.535013 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:41.535013 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:41.535013 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:41.535013 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:41.535013 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:41.535013 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:57:41.535013 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:57:41.535013 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 16:57:41.535013 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:57:41.535013 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 649/1298] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 23 16:57:41 2022] GET /identity/v3/services/placement => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 16:57:41.541502 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb56b0-8205-4b14-a57e-88a017b9a3cd None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:41.573059 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:41.683861 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4eb56b0-8205-4b14-a57e-88a017b9a3cd None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:41.720382 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd34702-3d7c-4e90-b999-e5371b4556a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:41.721036 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd34702-3d7c-4e90-b999-e5371b4556a3 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:41.721311 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd34702-3d7c-4e90-b999-e5371b4556a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:41.721577 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd34702-3d7c-4e90-b999-e5371b4556a3 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:41.722267 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd34702-3d7c-4e90-b999-e5371b4556a3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:41.722564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd34702-3d7c-4e90-b999-e5371b4556a3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:41.723970 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd34702-3d7c-4e90-b999-e5371b4556a3 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:41.729782 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 650/1299] 10.222.0.13 () {56 vars in 1116 bytes} [Mon May 23 16:57:41 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:57:41.735870 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda2722-3d4b-4f31-8668-a63abe1002e5 None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:41.771066 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:41.838698 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bda2722-3d4b-4f31-8668-a63abe1002e5 None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:41.873471 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a17696-eddf-4c1f-9b8e-2d5d665a23b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:41.874131 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a17696-eddf-4c1f-9b8e-2d5d665a23b1 None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:41.874413 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a17696-eddf-4c1f-9b8e-2d5d665a23b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:41.874686 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a17696-eddf-4c1f-9b8e-2d5d665a23b1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:41.875498 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a17696-eddf-4c1f-9b8e-2d5d665a23b1 None admin] RBAC: Adding query filter params (interface=public, service_id=3cce37bf20f9435c9c0e1d39a4c6054d, region_id=RegionOne) {{(pid=80465) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:41.875807 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a17696-eddf-4c1f-9b8e-2d5d665a23b1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:41.877196 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a17696-eddf-4c1f-9b8e-2d5d665a23b1 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:41.882773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 650/1300] 10.222.0.13 () {56 vars in 1251 bytes} [Mon May 23 16:57:41 2022] GET /identity/v3/endpoints?service_id=3cce37bf20f9435c9c0e1d39a4c6054d&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 16:57:41.888820 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd34702-3d7c-4e90-b999-e5371b4556a3 None admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:41.981136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:42.001122 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd34702-3d7c-4e90-b999-e5371b4556a3 None admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:42.037117 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee2b63d-a152-4a5c-9c08-42548086816c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:42.037772 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2b63d-a152-4a5c-9c08-42548086816c None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:42.038053 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2b63d-a152-4a5c-9c08-42548086816c None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:42.038326 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee2b63d-a152-4a5c-9c08-42548086816c None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:42.039030 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee2b63d-a152-4a5c-9c08-42548086816c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:42.040409 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee2b63d-a152-4a5c-9c08-42548086816c None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:42.047363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 651/1301] 10.222.0.13 () {56 vars in 1087 bytes} [Mon May 23 16:57:41 2022] GET /identity/v3/services => generated 2191 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 16:57:44.221552 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182c1cd-1bf2-4c38-88c3-e12ab2378058 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:44.222107 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182c1cd-1bf2-4c38-88c3-e12ab2378058 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:44.222107 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4182c1cd-1bf2-4c38-88c3-e12ab2378058 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:44.223465 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 651/1302] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:57:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:57:44.229866 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32c5ef-d0f6-4c0c-935a-f027acebdbaa None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:44.230621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32c5ef-d0f6-4c0c-935a-f027acebdbaa None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:44.230717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32c5ef-d0f6-4c0c-935a-f027acebdbaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:44.254367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-2d32c5ef-d0f6-4c0c-935a-f027acebdbaa None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:57:44.255592 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-2d32c5ef-d0f6-4c0c-935a-f027acebdbaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:57:44.328865 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 652/1303] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:57:44 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 16:57:44.351992 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182c1cd-1bf2-4c38-88c3-e12ab2378058 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:44.460890 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:44.468910 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4182c1cd-1bf2-4c38-88c3-e12ab2378058 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:44.505477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af2e87-5ae6-40de-83b8-dc9327b86bcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:44.506266 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af2e87-5ae6-40de-83b8-dc9327b86bcd None admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:44.506544 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af2e87-5ae6-40de-83b8-dc9327b86bcd None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:44.506814 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52af2e87-5ae6-40de-83b8-dc9327b86bcd None admin] PATH_INFO: `/v3/services/placement` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:44.507487 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af2e87-5ae6-40de-83b8-dc9327b86bcd None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:44.508861 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52af2e87-5ae6-40de-83b8-dc9327b86bcd None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:44.520844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application [None req-52af2e87-5ae6-40de-83b8-dc9327b86bcd None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 23 16:57:44.520844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:44.520844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 16:57:44.520844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 16:57:44.520844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 16:57:44.520844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 16:57:44.520844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 16:57:44.520844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:57:44.520844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 16:57:44.520844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:57:44.520844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 16:57:44.520844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:57:44.520844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 16:57:44.520844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 16:57:44.520844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 16:57:44.520844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 16:57:44.520844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 16:57:44.521990 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 16:57:44.521990 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 16:57:44.521990 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 16:57:44.521990 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 16:57:44.521990 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 16:57:44.521990 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 16:57:44.521990 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 16:57:44.521990 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 16:57:44.521990 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 16:57:44.521990 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 16:57:44.521990 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 16:57:44.521990 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 16:57:44.521990 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 16:57:44.521990 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 16:57:44.521990 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 16:57:44.521990 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.get_or_create( May 23 16:57:44.523165 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 16:57:44.523165 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application with Lock( May 23 16:57:44.523165 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 16:57:44.523165 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self._enter() May 23 16:57:44.523165 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 16:57:44.523165 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 16:57:44.523165 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 16:57:44.523165 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application return self.creator() May 23 16:57:44.523165 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 16:57:44.523165 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application created_value = creator( May 23 16:57:44.523165 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 16:57:44.523165 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 16:57:44.523165 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 16:57:44.523165 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: ERROR keystone.server.flask.application May 23 16:57:44.523165 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 652/1304] 10.222.0.13 () {56 vars in 1117 bytes} [Mon May 23 16:57:44 2022] GET /identity/v3/services/placement => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 16:57:44.528488 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32c5ef-d0f6-4c0c-935a-f027acebdbaa None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:44.609135 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:44.643740 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32c5ef-d0f6-4c0c-935a-f027acebdbaa None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:44.679578 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb613c32-f489-4269-bc00-76a9c868c475 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:44.680235 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb613c32-f489-4269-bc00-76a9c868c475 None admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:44.680514 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb613c32-f489-4269-bc00-76a9c868c475 None admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:44.680792 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb613c32-f489-4269-bc00-76a9c868c475 None admin] PATH_INFO: `/v3/services` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:44.681630 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb613c32-f489-4269-bc00-76a9c868c475 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 16:57:44.681940 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb613c32-f489-4269-bc00-76a9c868c475 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:44.683359 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb613c32-f489-4269-bc00-76a9c868c475 None admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:44.688626 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 653/1305] 10.222.0.13 () {56 vars in 1116 bytes} [Mon May 23 16:57:44 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 16:57:44.694779 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af2e87-5ae6-40de-83b8-dc9327b86bcd None admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:57:44.799793 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52af2e87-5ae6-40de-83b8-dc9327b86bcd None admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:57:44.835843 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f0f8c2-d102-496c-9540-ff8d8f1806d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:57:44.836476 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0f8c2-d102-496c-9540-ff8d8f1806d2 None admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:57:44.836752 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0f8c2-d102-496c-9540-ff8d8f1806d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:57:44.837027 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f0f8c2-d102-496c-9540-ff8d8f1806d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:57:44.837959 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f0f8c2-d102-496c-9540-ff8d8f1806d2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:57:44.839360 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f0f8c2-d102-496c-9540-ff8d8f1806d2 None admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:57:44.862178 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 653/1306] 10.222.0.13 () {60 vars in 1144 bytes} [Mon May 23 16:57:44 2022] POST /identity/v3/endpoints => generated 331 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 16:57:54.685430 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=30 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:57:54.857603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:58:49.193363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852cad1-6757-4c69-ba04-d016c485305c None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:58:49.193931 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852cad1-6757-4c69-ba04-d016c485305c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:58:49.193931 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e852cad1-6757-4c69-ba04-d016c485305c None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:58:49.199633 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 654/1307] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:58:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:58:49.201318 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ac1d3-43f6-44bf-b02e-5d9c4c72ea43 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:58:49.202412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ac1d3-43f6-44bf-b02e-5d9c4c72ea43 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:58:49.203016 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1ac1d3-43f6-44bf-b02e-5d9c4c72ea43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:58:49.305653 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-3d1ac1d3-43f6-44bf-b02e-5d9c4c72ea43 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:58:49.306882 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-3d1ac1d3-43f6-44bf-b02e-5d9c4c72ea43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:58:49.422459 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 654/1308] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:58:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:58:49.482415 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67496059-6ba4-4529-be1e-c07a99d1bedd None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:58:49.482887 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67496059-6ba4-4529-be1e-c07a99d1bedd None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:58:49.483621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67496059-6ba4-4529-be1e-c07a99d1bedd None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:58:49.486850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 655/1309] 10.222.0.13 () {54 vars in 964 bytes} [Mon May 23 16:58:49 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:58:49.494387 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ccfaf-e631-4424-9c46-381cda662343 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:58:49.494679 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ccfaf-e631-4424-9c46-381cda662343 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:58:49.495055 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352ccfaf-e631-4424-9c46-381cda662343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:58:49.604645 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-352ccfaf-e631-4424-9c46-381cda662343 None None] MFA Rules not processed for user `fd070e7581f04f4abcbb8b6bc231a8c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:58:49.605837 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-352ccfaf-e631-4424-9c46-381cda662343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:58:49.627731 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:58:49.742950 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 655/1310] 10.222.0.13 () {58 vars in 1061 bytes} [Mon May 23 16:58:49 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:58:49.756647 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67496059-6ba4-4529-be1e-c07a99d1bedd None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:58:49.758765 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-67496059-6ba4-4529-be1e-c07a99d1bedd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:58:49.883193 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67496059-6ba4-4529-be1e-c07a99d1bedd None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:58:49.960780 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ed7a16-cfee-4eb9-b60e-ecff00db65c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:58:49.961721 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed7a16-cfee-4eb9-b60e-ecff00db65c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:58:49.962143 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed7a16-cfee-4eb9-b60e-ecff00db65c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:58:49.962638 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ed7a16-cfee-4eb9-b60e-ecff00db65c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:58:49.964585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-98ed7a16-cfee-4eb9-b60e-ecff00db65c5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:58:50.025300 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:58:50.088477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed7a16-cfee-4eb9-b60e-ecff00db65c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:58:50.090352 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ed7a16-cfee-4eb9-b60e-ecff00db65c5 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:58:50.111261 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 656/1311] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 16:58:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:58:52.625235 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89d604-bdac-4c66-8ada-766dea06ae8a None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:58:52.625728 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89d604-bdac-4c66-8ada-766dea06ae8a None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:58:52.625815 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc89d604-bdac-4c66-8ada-766dea06ae8a None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:58:52.631312 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 656/1312] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:58:52 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:58:52.633218 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990d257-5bcb-4a66-8c8f-0ee8d7434f33 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:58:52.633630 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990d257-5bcb-4a66-8c8f-0ee8d7434f33 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:58:52.633840 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2990d257-5bcb-4a66-8c8f-0ee8d7434f33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:58:52.717367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-2990d257-5bcb-4a66-8c8f-0ee8d7434f33 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:58:52.718637 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-2990d257-5bcb-4a66-8c8f-0ee8d7434f33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:58:52.822796 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 657/1313] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:58:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:58:52.858104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89d604-bdac-4c66-8ada-766dea06ae8a None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:58:52.859801 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-bc89d604-bdac-4c66-8ada-766dea06ae8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:58:52.972607 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc89d604-bdac-4c66-8ada-766dea06ae8a None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:58:53.046643 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:58:53.049466 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0198aeb-c808-4888-bf74-3c6cd070c9ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:58:53.050170 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0198aeb-c808-4888-bf74-3c6cd070c9ae service neutron] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:58:53.050625 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0198aeb-c808-4888-bf74-3c6cd070c9ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:58:53.051031 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0198aeb-c808-4888-bf74-3c6cd070c9ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:58:53.053502 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-f0198aeb-c808-4888-bf74-3c6cd070c9ae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:58:53.169773 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0198aeb-c808-4888-bf74-3c6cd070c9ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:58:53.171253 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0198aeb-c808-4888-bf74-3c6cd070c9ae service neutron] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:58:53.189871 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 657/1314] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 16:58:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:58:55.709541 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d325a-19af-4f64-a7c7-a99d5c84e7ca None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:58:55.710385 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d325a-19af-4f64-a7c7-a99d5c84e7ca None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:58:55.710385 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186d325a-19af-4f64-a7c7-a99d5c84e7ca None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:58:55.713183 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 658/1315] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:58:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:58:55.721845 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264bca9-4fd5-4c27-a8f5-696b8e91d300 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:58:55.722271 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264bca9-4fd5-4c27-a8f5-696b8e91d300 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:58:55.722400 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264bca9-4fd5-4c27-a8f5-696b8e91d300 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:58:55.791934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-0264bca9-4fd5-4c27-a8f5-696b8e91d300 None None] MFA Rules not processed for user `99a4070a60cc4ae8a1981ac1c10668fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:58:55.793064 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-0264bca9-4fd5-4c27-a8f5-696b8e91d300 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:58:55.850543 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:58:55.898145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 658/1316] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:58:55 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:58:55.940538 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d325a-19af-4f64-a7c7-a99d5c84e7ca None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:58:55.942049 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-186d325a-19af-4f64-a7c7-a99d5c84e7ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:58:55.953231 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:58:56.049775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186d325a-19af-4f64-a7c7-a99d5c84e7ca None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:58:56.129655 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da417c68-82a1-4d39-ab6a-536f2b667b6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:58:56.130295 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da417c68-82a1-4d39-ab6a-536f2b667b6a service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:58:56.130570 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da417c68-82a1-4d39-ab6a-536f2b667b6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:58:56.130840 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da417c68-82a1-4d39-ab6a-536f2b667b6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:58:56.132941 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-da417c68-82a1-4d39-ab6a-536f2b667b6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:58:56.203679 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:58:56.252230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da417c68-82a1-4d39-ab6a-536f2b667b6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:58:56.253998 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da417c68-82a1-4d39-ab6a-536f2b667b6a service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:58:56.274953 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 659/1317] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 16:58:55 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 16:58:59.043940 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4fb422-4c65-4e5a-bede-915bf1dc1c9d None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:58:59.044505 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4fb422-4c65-4e5a-bede-915bf1dc1c9d None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:58:59.044505 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4fb422-4c65-4e5a-bede-915bf1dc1c9d None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:58:59.045877 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 659/1318] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:58:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:58:59.051361 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dbb594-4db4-473c-831f-d7001cc731e5 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:58:59.051805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dbb594-4db4-473c-831f-d7001cc731e5 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:58:59.051906 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dbb594-4db4-473c-831f-d7001cc731e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:58:59.119921 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-72dbb594-4db4-473c-831f-d7001cc731e5 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:58:59.121138 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-72dbb594-4db4-473c-831f-d7001cc731e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:58:59.219382 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 660/1319] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:58:59 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:58:59.280491 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c78ba0-24ec-4bba-9f5b-fa06da76a809 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:58:59.281058 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c78ba0-24ec-4bba-9f5b-fa06da76a809 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:58:59.281523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c78ba0-24ec-4bba-9f5b-fa06da76a809 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:58:59.284176 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 660/1320] 10.222.0.13 () {54 vars in 964 bytes} [Mon May 23 16:58:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:58:59.290874 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00749f3d-1349-4abf-8b47-3bb5ded4bdc5 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:58:59.291314 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00749f3d-1349-4abf-8b47-3bb5ded4bdc5 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:58:59.291720 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00749f3d-1349-4abf-8b47-3bb5ded4bdc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:58:59.317317 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:58:59.381622 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-00749f3d-1349-4abf-8b47-3bb5ded4bdc5 None None] MFA Rules not processed for user `fd070e7581f04f4abcbb8b6bc231a8c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:58:59.383259 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-00749f3d-1349-4abf-8b47-3bb5ded4bdc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:58:59.461367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 661/1321] 10.222.0.13 () {58 vars in 1061 bytes} [Mon May 23 16:58:59 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:58:59.473603 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c78ba0-24ec-4bba-9f5b-fa06da76a809 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:58:59.475197 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-37c78ba0-24ec-4bba-9f5b-fa06da76a809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:58:59.585881 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c78ba0-24ec-4bba-9f5b-fa06da76a809 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:58:59.665374 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc8a9ce-5892-4c3d-8882-23cd9ec7718a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:58:59.666010 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8a9ce-5892-4c3d-8882-23cd9ec7718a service neutron] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:58:59.666286 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8a9ce-5892-4c3d-8882-23cd9ec7718a service neutron] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:58:59.666547 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc8a9ce-5892-4c3d-8882-23cd9ec7718a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:58:59.668192 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-5dc8a9ce-5892-4c3d-8882-23cd9ec7718a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:58:59.675823 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:58:59.789908 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8a9ce-5892-4c3d-8882-23cd9ec7718a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:58:59.791401 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc8a9ce-5892-4c3d-8882-23cd9ec7718a service neutron] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:58:59.809617 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 661/1322] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 16:58:59 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:59:03.099773 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a39fc6-025e-4d4e-a915-5244b6b7fe13 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:03.100366 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a39fc6-025e-4d4e-a915-5244b6b7fe13 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:03.100366 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57a39fc6-025e-4d4e-a915-5244b6b7fe13 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:03.101727 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 662/1323] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:59:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:59:03.107162 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc30cf7-8f5c-49d0-8fb1-8cb0da64bf40 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:03.107601 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc30cf7-8f5c-49d0-8fb1-8cb0da64bf40 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:03.107707 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc30cf7-8f5c-49d0-8fb1-8cb0da64bf40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:03.178889 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-ebc30cf7-8f5c-49d0-8fb1-8cb0da64bf40 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:59:03.180347 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-ebc30cf7-8f5c-49d0-8fb1-8cb0da64bf40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:03.238990 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:59:03.277958 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 662/1324] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:59:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:59:03.312560 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a39fc6-025e-4d4e-a915-5244b6b7fe13 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:59:03.314064 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-57a39fc6-025e-4d4e-a915-5244b6b7fe13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:03.420909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57a39fc6-025e-4d4e-a915-5244b6b7fe13 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:59:03.433477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:59:03.499997 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b73fe12-7d16-4b63-96ed-4737f0120dd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:59:03.500648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73fe12-7d16-4b63-96ed-4737f0120dd4 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:03.500924 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73fe12-7d16-4b63-96ed-4737f0120dd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:03.501179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b73fe12-7d16-4b63-96ed-4737f0120dd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:03.503136 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8b73fe12-7d16-4b63-96ed-4737f0120dd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:03.616437 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b73fe12-7d16-4b63-96ed-4737f0120dd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:59:03.617926 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b73fe12-7d16-4b63-96ed-4737f0120dd4 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:59:03.637295 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 663/1325] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 16:59:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 16:59:05.876334 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826156c-cbb5-4a0e-802c-1c5ccce92c89 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:05.876895 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826156c-cbb5-4a0e-802c-1c5ccce92c89 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:05.876895 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7826156c-cbb5-4a0e-802c-1c5ccce92c89 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:05.878592 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 663/1326] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:59:05.884241 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ff4a3-57d5-45f5-8791-5c1c106136aa None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:05.884713 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ff4a3-57d5-45f5-8791-5c1c106136aa None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:05.884808 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896ff4a3-57d5-45f5-8791-5c1c106136aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:05.935641 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:59:05.954686 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-896ff4a3-57d5-45f5-8791-5c1c106136aa None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:59:05.955918 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-896ff4a3-57d5-45f5-8791-5c1c106136aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:06.057729 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 664/1327] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:59:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:59:06.093986 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826156c-cbb5-4a0e-802c-1c5ccce92c89 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:59:06.095547 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-7826156c-cbb5-4a0e-802c-1c5ccce92c89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:06.204346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7826156c-cbb5-4a0e-802c-1c5ccce92c89 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:59:06.282445 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3164065-1500-4d6b-a77e-dea01c436c91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:59:06.283132 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3164065-1500-4d6b-a77e-dea01c436c91 service neutron] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:06.283416 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3164065-1500-4d6b-a77e-dea01c436c91 service neutron] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:06.283689 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3164065-1500-4d6b-a77e-dea01c436c91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:06.285323 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-e3164065-1500-4d6b-a77e-dea01c436c91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:06.302889 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:59:06.400891 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3164065-1500-4d6b-a77e-dea01c436c91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:59:06.402414 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3164065-1500-4d6b-a77e-dea01c436c91 service neutron] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:59:06.422244 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 664/1328] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 16:59:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:59:08.649668 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a902e3-0b52-453b-a21d-91c25a05ff2b None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:08.650278 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a902e3-0b52-453b-a21d-91c25a05ff2b None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:08.650278 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a902e3-0b52-453b-a21d-91c25a05ff2b None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:08.651649 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 665/1329] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:59:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:59:08.657033 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77aac91-34fa-477a-8ae5-c5bb8a74a089 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:08.657453 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77aac91-34fa-477a-8ae5-c5bb8a74a089 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:08.657567 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77aac91-34fa-477a-8ae5-c5bb8a74a089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:08.723189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-f77aac91-34fa-477a-8ae5-c5bb8a74a089 None None] MFA Rules not processed for user `99a4070a60cc4ae8a1981ac1c10668fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:59:08.724362 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-f77aac91-34fa-477a-8ae5-c5bb8a74a089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:08.791375 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:59:08.824379 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 665/1330] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:59:08 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:59:08.860865 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a902e3-0b52-453b-a21d-91c25a05ff2b None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:59:08.862437 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-93a902e3-0b52-453b-a21d-91c25a05ff2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:08.973640 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a902e3-0b52-453b-a21d-91c25a05ff2b None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:59:08.993901 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:59:09.051796 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed91354-92dc-464f-91d5-456507d14a0e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:59:09.052364 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed91354-92dc-464f-91d5-456507d14a0e service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:09.052641 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed91354-92dc-464f-91d5-456507d14a0e service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:09.052916 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed91354-92dc-464f-91d5-456507d14a0e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:09.054620 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8ed91354-92dc-464f-91d5-456507d14a0e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:09.169431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed91354-92dc-464f-91d5-456507d14a0e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:59:09.170891 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed91354-92dc-464f-91d5-456507d14a0e service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:59:09.189391 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 666/1331] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 16:59:08 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:59:11.677109 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f07df-789f-47d7-9009-4593693425eb None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:11.677624 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f07df-789f-47d7-9009-4593693425eb None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:11.677624 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967f07df-789f-47d7-9009-4593693425eb None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:11.678945 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 666/1332] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:59:11.685399 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a93960-eef4-4c72-b7e2-168c766a0266 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:11.686120 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a93960-eef4-4c72-b7e2-168c766a0266 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:11.686272 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a93960-eef4-4c72-b7e2-168c766a0266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:11.754289 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:59:11.757163 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-74a93960-eef4-4c72-b7e2-168c766a0266 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:59:11.759034 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-74a93960-eef4-4c72-b7e2-168c766a0266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:11.863148 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 667/1333] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:59:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:59:11.899687 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f07df-789f-47d7-9009-4593693425eb None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:59:11.901232 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-967f07df-789f-47d7-9009-4593693425eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:12.016774 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967f07df-789f-47d7-9009-4593693425eb None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:59:12.098952 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c66448-7569-4e01-b481-bbd22f080d15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:59:12.099641 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c66448-7569-4e01-b481-bbd22f080d15 service neutron] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:12.099921 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c66448-7569-4e01-b481-bbd22f080d15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:12.100191 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c66448-7569-4e01-b481-bbd22f080d15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:12.102843 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-72c66448-7569-4e01-b481-bbd22f080d15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:12.135153 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:59:12.222643 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c66448-7569-4e01-b481-bbd22f080d15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:59:12.224165 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c66448-7569-4e01-b481-bbd22f080d15 service neutron] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:59:12.245312 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 667/1334] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 16:59:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:59:14.478394 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3063e-f99a-4c24-bc37-88449cd64073 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:14.479447 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3063e-f99a-4c24-bc37-88449cd64073 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:14.479447 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d3063e-f99a-4c24-bc37-88449cd64073 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:14.480379 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 668/1335] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:59:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 16:59:14.489121 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d171aa-9b2b-40c0-be1c-fd77434b0636 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:14.489832 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d171aa-9b2b-40c0-be1c-fd77434b0636 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:14.489938 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d171aa-9b2b-40c0-be1c-fd77434b0636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:14.567285 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-29d171aa-9b2b-40c0-be1c-fd77434b0636 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:59:14.568499 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-29d171aa-9b2b-40c0-be1c-fd77434b0636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:14.648590 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:59:14.671917 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 668/1336] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:59:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:59:14.711811 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3063e-f99a-4c24-bc37-88449cd64073 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:59:14.713369 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-51d3063e-f99a-4c24-bc37-88449cd64073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:14.828833 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d3063e-f99a-4c24-bc37-88449cd64073 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:59:14.858204 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:59:14.905663 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7608d15f-96d0-4b22-8921-cd94a350fc9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:59:14.906327 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608d15f-96d0-4b22-8921-cd94a350fc9c service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:14.906613 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608d15f-96d0-4b22-8921-cd94a350fc9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:14.906885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7608d15f-96d0-4b22-8921-cd94a350fc9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:14.908553 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7608d15f-96d0-4b22-8921-cd94a350fc9c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:15.033796 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608d15f-96d0-4b22-8921-cd94a350fc9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:59:15.035280 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7608d15f-96d0-4b22-8921-cd94a350fc9c service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:59:15.054006 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 669/1337] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 16:59:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 16:59:17.633288 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb3557-03bb-45d8-a3a2-d01c83d6192f None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:17.633840 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb3557-03bb-45d8-a3a2-d01c83d6192f None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:17.633840 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb3557-03bb-45d8-a3a2-d01c83d6192f None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:17.635264 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 669/1338] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:59:17.641909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571680cb-334b-43dc-b91e-8b8bcba239a5 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:17.642617 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571680cb-334b-43dc-b91e-8b8bcba239a5 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:17.642999 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571680cb-334b-43dc-b91e-8b8bcba239a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:17.724135 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-571680cb-334b-43dc-b91e-8b8bcba239a5 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:59:17.725335 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-571680cb-334b-43dc-b91e-8b8bcba239a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:17.734009 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:59:17.826964 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 670/1339] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:59:17 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:59:17.862786 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb3557-03bb-45d8-a3a2-d01c83d6192f None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:59:17.865188 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-f6fb3557-03bb-45d8-a3a2-d01c83d6192f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:17.986143 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fb3557-03bb-45d8-a3a2-d01c83d6192f None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:59:18.067969 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbcc52-b66e-4273-9373-3a6f52f294cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:59:18.068612 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbcc52-b66e-4273-9373-3a6f52f294cc service neutron] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:18.068886 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbcc52-b66e-4273-9373-3a6f52f294cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:18.069164 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbcc52-b66e-4273-9373-3a6f52f294cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:18.070874 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-76cbcc52-b66e-4273-9373-3a6f52f294cc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:18.106840 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:59:18.192029 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbcc52-b66e-4273-9373-3a6f52f294cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:59:18.193529 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbcc52-b66e-4273-9373-3a6f52f294cc service neutron] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:59:18.212024 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 670/1340] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 16:59:17 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:59:22.578357 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3af72-1e6e-411b-9e64-6299b187f7f3 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:22.578938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3af72-1e6e-411b-9e64-6299b187f7f3 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:22.578938 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3af72-1e6e-411b-9e64-6299b187f7f3 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:22.580455 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 671/1341] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:59:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:59:22.585830 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6e76f-dbcf-4ef7-a330-77c7553db0a1 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:22.586390 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6e76f-dbcf-4ef7-a330-77c7553db0a1 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:22.586390 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc6e76f-dbcf-4ef7-a330-77c7553db0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:22.666352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-9bc6e76f-dbcf-4ef7-a330-77c7553db0a1 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:59:22.667586 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-9bc6e76f-dbcf-4ef7-a330-77c7553db0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:22.765250 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=100 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:59:22.779558 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 671/1342] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:59:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:59:22.818973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e3af72-1e6e-411b-9e64-6299b187f7f3 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:59:22.821505 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a9e3af72-1e6e-411b-9e64-6299b187f7f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:22.923168 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e3af72-1e6e-411b-9e64-6299b187f7f3 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:59:22.927025 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeada4a3-cf1b-4d2a-bd0d-89a6f27c08e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:59:22.927650 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeada4a3-cf1b-4d2a-bd0d-89a6f27c08e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:22.927922 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeada4a3-cf1b-4d2a-bd0d-89a6f27c08e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:22.928186 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeada4a3-cf1b-4d2a-bd0d-89a6f27c08e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:22.937799 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeada4a3-cf1b-4d2a-bd0d-89a6f27c08e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:59:22.939267 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeada4a3-cf1b-4d2a-bd0d-89a6f27c08e6 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:59:22.948657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 672/1343] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 16:59:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:59:25.733784 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8299168-2f35-432b-a6fe-57b33bfbac1f None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:25.734313 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8299168-2f35-432b-a6fe-57b33bfbac1f None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:25.734313 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8299168-2f35-432b-a6fe-57b33bfbac1f None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:25.735663 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 672/1344] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:59:25.741132 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b50f2-2c3d-4597-ad1e-12f2f3b5b5a2 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:25.741599 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b50f2-2c3d-4597-ad1e-12f2f3b5b5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:25.741704 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6b50f2-2c3d-4597-ad1e-12f2f3b5b5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:25.771148 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-0f6b50f2-2c3d-4597-ad1e-12f2f3b5b5a2 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:59:25.772331 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-0f6b50f2-2c3d-4597-ad1e-12f2f3b5b5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:25.800991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:59:25.853499 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 673/1345] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:59:25 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:59:25.890302 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8299168-2f35-432b-a6fe-57b33bfbac1f None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:59:25.892226 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-c8299168-2f35-432b-a6fe-57b33bfbac1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:25.906864 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8299168-2f35-432b-a6fe-57b33bfbac1f None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:59:25.910821 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c036d4a9-2d88-4917-b12d-f47526413268 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:59:25.911521 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036d4a9-2d88-4917-b12d-f47526413268 service neutron] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:25.911804 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036d4a9-2d88-4917-b12d-f47526413268 service neutron] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:25.912075 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c036d4a9-2d88-4917-b12d-f47526413268 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:25.922216 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036d4a9-2d88-4917-b12d-f47526413268 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:59:25.923696 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c036d4a9-2d88-4917-b12d-f47526413268 service neutron] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:59:25.933431 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 673/1346] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 16:59:25 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:59:30.355377 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca62ea-8a56-48a1-82b6-144f80778f47 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:30.355993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca62ea-8a56-48a1-82b6-144f80778f47 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:30.355993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cca62ea-8a56-48a1-82b6-144f80778f47 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:30.357318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 674/1347] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:59:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:59:30.362624 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b58aa6-8850-4f72-a0e8-3d5bce8b311b None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:30.363123 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b58aa6-8850-4f72-a0e8-3d5bce8b311b None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:30.363123 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b58aa6-8850-4f72-a0e8-3d5bce8b311b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:30.382768 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-d7b58aa6-8850-4f72-a0e8-3d5bce8b311b None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:59:30.383934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-d7b58aa6-8850-4f72-a0e8-3d5bce8b311b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:30.393500 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 674/1348] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:59:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:59:30.429818 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca62ea-8a56-48a1-82b6-144f80778f47 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:59:30.441546 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cca62ea-8a56-48a1-82b6-144f80778f47 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:59:30.445837 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1928a3ab-be19-415d-93b2-0c1d248c598d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:59:30.446483 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928a3ab-be19-415d-93b2-0c1d248c598d service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:30.446738 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928a3ab-be19-415d-93b2-0c1d248c598d service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:30.447064 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1928a3ab-be19-415d-93b2-0c1d248c598d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:30.456876 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928a3ab-be19-415d-93b2-0c1d248c598d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:59:30.458331 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1928a3ab-be19-415d-93b2-0c1d248c598d service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:59:30.471515 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 675/1349] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 16:59:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:59:35.909935 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52333a41-68ce-4896-bb6f-edc6a3040516 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:35.910453 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52333a41-68ce-4896-bb6f-edc6a3040516 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:35.910561 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52333a41-68ce-4896-bb6f-edc6a3040516 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:35.912130 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 675/1350] 10.222.0.13 () {54 vars in 921 bytes} [Mon May 23 16:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:59:35.917523 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b243b81-e4f4-4996-b1ec-f7ef3b5385d4 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:35.918359 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b243b81-e4f4-4996-b1ec-f7ef3b5385d4 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:35.918858 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b243b81-e4f4-4996-b1ec-f7ef3b5385d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:35.995105 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-9b243b81-e4f4-4996-b1ec-f7ef3b5385d4 None None] MFA Rules not processed for user `f181ebdfcd544b54af7e6f9c118fe252`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:59:36.008763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9b243b81-e4f4-4996-b1ec-f7ef3b5385d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:36.101088 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 676/1351] 10.222.0.13 () {58 vars in 1018 bytes} [Mon May 23 16:59:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:59:40.376273 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=13 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:59:40.396029 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed0ec1-d4f0-421a-b0ba-81aca651d855 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:40.396314 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed0ec1-d4f0-421a-b0ba-81aca651d855 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:40.396634 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ed0ec1-d4f0-421a-b0ba-81aca651d855 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:40.398297 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 676/1352] 10.222.0.13 () {54 vars in 921 bytes} [Mon May 23 16:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:59:40.403744 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb5a27-18e8-4b05-b201-15026e2d070d None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:40.404656 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb5a27-18e8-4b05-b201-15026e2d070d None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:40.405143 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50eb5a27-18e8-4b05-b201-15026e2d070d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:40.425345 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-50eb5a27-18e8-4b05-b201-15026e2d070d None None] MFA Rules not processed for user `f181ebdfcd544b54af7e6f9c118fe252`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:59:40.426535 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-50eb5a27-18e8-4b05-b201-15026e2d070d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:40.436364 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 677/1353] 10.222.0.13 () {58 vars in 1018 bytes} [Mon May 23 16:59:40 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:59:41.718086 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec3f8a-206b-49e1-9821-cbc82fbf617d None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:41.718648 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec3f8a-206b-49e1-9821-cbc82fbf617d None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:41.718741 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfec3f8a-206b-49e1-9821-cbc82fbf617d None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:41.720427 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 677/1354] 10.222.0.13 () {54 vars in 921 bytes} [Mon May 23 16:59:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:59:41.725744 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aee714-9cbf-4347-9fc6-4d169ef2bffb None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:41.726216 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aee714-9cbf-4347-9fc6-4d169ef2bffb None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:41.726309 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72aee714-9cbf-4347-9fc6-4d169ef2bffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:41.745460 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-72aee714-9cbf-4347-9fc6-4d169ef2bffb None None] MFA Rules not processed for user `f181ebdfcd544b54af7e6f9c118fe252`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:59:41.746561 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-72aee714-9cbf-4347-9fc6-4d169ef2bffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:41.755984 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 678/1355] 10.222.0.13 () {58 vars in 1018 bytes} [Mon May 23 16:59:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:59:45.693750 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e9b3d-19fe-4bb3-93e6-8b5b1adf27b6 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:45.694303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e9b3d-19fe-4bb3-93e6-8b5b1adf27b6 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:45.694303 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e9b3d-19fe-4bb3-93e6-8b5b1adf27b6 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:45.695934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 678/1356] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:59:45.701372 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4458c-4a6e-4b9e-b17d-9a7349b43515 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:45.701825 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4458c-4a6e-4b9e-b17d-9a7349b43515 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:45.701922 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a4458c-4a6e-4b9e-b17d-9a7349b43515 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:45.721135 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-a5a4458c-4a6e-4b9e-b17d-9a7349b43515 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:59:45.722307 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-a5a4458c-4a6e-4b9e-b17d-9a7349b43515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:45.730491 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 679/1357] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:59:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:59:45.785246 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e84b89-1447-42da-8539-efd4dad29d81 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:45.786103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e84b89-1447-42da-8539-efd4dad29d81 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:45.786942 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e84b89-1447-42da-8539-efd4dad29d81 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:45.790139 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 679/1358] 10.222.0.13 () {54 vars in 961 bytes} [Mon May 23 16:59:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:59:45.800413 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e46a25-d34c-475c-b5d0-b5d83b93b40c None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:45.801156 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e46a25-d34c-475c-b5d0-b5d83b93b40c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:45.802396 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e46a25-d34c-475c-b5d0-b5d83b93b40c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:45.848670 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:59:45.857330 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-e9e46a25-d34c-475c-b5d0-b5d83b93b40c None None] MFA Rules not processed for user `225c7b385f12445fab86b3b5606b1da1`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:59:45.858517 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-e9e46a25-d34c-475c-b5d0-b5d83b93b40c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:45.949075 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 680/1359] 10.222.0.13 () {58 vars in 1058 bytes} [Mon May 23 16:59:45 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:59:45.956787 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e84b89-1447-42da-8539-efd4dad29d81 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:59:45.973912 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e84b89-1447-42da-8539-efd4dad29d81 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:59:45.980539 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3d77eb-f7ed-4d85-b00e-5bb72d777972 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:59:45.981344 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d77eb-f7ed-4d85-b00e-5bb72d777972 service nova] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:45.981963 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d77eb-f7ed-4d85-b00e-5bb72d777972 service nova] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:45.982417 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3d77eb-f7ed-4d85-b00e-5bb72d777972 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:45.991715 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d77eb-f7ed-4d85-b00e-5bb72d777972 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:59:45.994381 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3d77eb-f7ed-4d85-b00e-5bb72d777972 service nova] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:59:46.006259 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 680/1360] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 16:59:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:59:48.437137 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c1223-a1ca-4626-903c-abf727bc31ea None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:48.438360 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c1223-a1ca-4626-903c-abf727bc31ea None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:48.438868 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924c1223-a1ca-4626-903c-abf727bc31ea None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:48.441268 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 681/1361] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:59:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:59:48.448400 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c845a-0b1c-4904-82c1-0bab13ed3d07 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:48.449072 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c845a-0b1c-4904-82c1-0bab13ed3d07 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:48.449072 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972c845a-0b1c-4904-82c1-0bab13ed3d07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:48.483822 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-972c845a-0b1c-4904-82c1-0bab13ed3d07 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:59:48.486444 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-972c845a-0b1c-4904-82c1-0bab13ed3d07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:48.501514 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 681/1362] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:59:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:59:48.545207 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924c1223-a1ca-4626-903c-abf727bc31ea None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:59:48.560185 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924c1223-a1ca-4626-903c-abf727bc31ea None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:59:48.565943 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262b403d-be83-4d6d-a408-384638c42cb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:59:48.566578 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b403d-be83-4d6d-a408-384638c42cb8 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:48.566856 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b403d-be83-4d6d-a408-384638c42cb8 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:48.567156 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262b403d-be83-4d6d-a408-384638c42cb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:48.575648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262b403d-be83-4d6d-a408-384638c42cb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:59:48.577243 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262b403d-be83-4d6d-a408-384638c42cb8 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:59:48.586943 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 682/1363] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 16:59:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:59:51.139566 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff1a0e-ae5a-485b-bd33-faaa5bcd7599 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:51.140070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff1a0e-ae5a-485b-bd33-faaa5bcd7599 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:51.140070 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ff1a0e-ae5a-485b-bd33-faaa5bcd7599 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:51.141705 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 682/1364] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:59:51.146969 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b65b7-ed04-415c-b1f2-e7685ce777e9 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:51.147789 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b65b7-ed04-415c-b1f2-e7685ce777e9 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:51.148289 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860b65b7-ed04-415c-b1f2-e7685ce777e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:51.170395 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-860b65b7-ed04-415c-b1f2-e7685ce777e9 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:59:51.170395 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-860b65b7-ed04-415c-b1f2-e7685ce777e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:51.193397 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 683/1365] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:59:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:59:51.234705 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085c4e0-9e66-438c-bac5-a5b0c310f30e None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:51.235051 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085c4e0-9e66-438c-bac5-a5b0c310f30e None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:51.235297 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6085c4e0-9e66-438c-bac5-a5b0c310f30e None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:51.236655 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 683/1366] 10.222.0.13 () {54 vars in 961 bytes} [Mon May 23 16:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:59:51.241917 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a61402-08b4-4041-b1a5-e5d8851d7977 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:51.242198 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a61402-08b4-4041-b1a5-e5d8851d7977 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:51.242474 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a61402-08b4-4041-b1a5-e5d8851d7977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:51.261331 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-06a61402-08b4-4041-b1a5-e5d8851d7977 None None] MFA Rules not processed for user `225c7b385f12445fab86b3b5606b1da1`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:59:51.262562 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-06a61402-08b4-4041-b1a5-e5d8851d7977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:51.272618 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 684/1367] 10.222.0.13 () {58 vars in 1058 bytes} [Mon May 23 16:59:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:59:51.281297 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085c4e0-9e66-438c-bac5-a5b0c310f30e None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:59:51.295752 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6085c4e0-9e66-438c-bac5-a5b0c310f30e None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:59:51.301403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4a1d2b-ecf7-40d4-9af7-4bf3e1d59ad6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:59:51.302040 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a1d2b-ecf7-40d4-9af7-4bf3e1d59ad6 service nova] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:51.302320 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a1d2b-ecf7-40d4-9af7-4bf3e1d59ad6 service nova] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:51.302591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4a1d2b-ecf7-40d4-9af7-4bf3e1d59ad6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:51.310989 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4a1d2b-ecf7-40d4-9af7-4bf3e1d59ad6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:59:51.312624 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4a1d2b-ecf7-40d4-9af7-4bf3e1d59ad6 service nova] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:59:51.322208 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 684/1368] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 16:59:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:59:53.600577 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34927f48-9b57-4a1c-92e2-af99b10e2033 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:53.601237 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34927f48-9b57-4a1c-92e2-af99b10e2033 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:53.601237 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34927f48-9b57-4a1c-92e2-af99b10e2033 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:53.602753 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 685/1369] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:59:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:59:53.608465 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82941918-ecdd-46db-a371-c23f081aaeeb None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:53.608938 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82941918-ecdd-46db-a371-c23f081aaeeb None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:53.609026 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82941918-ecdd-46db-a371-c23f081aaeeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:53.627200 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-82941918-ecdd-46db-a371-c23f081aaeeb None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:59:53.628371 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-82941918-ecdd-46db-a371-c23f081aaeeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:53.637275 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 685/1370] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:59:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:59:53.683986 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34927f48-9b57-4a1c-92e2-af99b10e2033 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:59:53.701855 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34927f48-9b57-4a1c-92e2-af99b10e2033 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:59:53.708912 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ba2bf0-7add-473b-a738-e0dd445ed513 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:59:53.709568 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba2bf0-7add-473b-a738-e0dd445ed513 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:53.709850 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba2bf0-7add-473b-a738-e0dd445ed513 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:53.710124 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ba2bf0-7add-473b-a738-e0dd445ed513 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:53.720275 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba2bf0-7add-473b-a738-e0dd445ed513 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:59:53.722327 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ba2bf0-7add-473b-a738-e0dd445ed513 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:59:53.732754 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 686/1371] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 16:59:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:59:53.871690 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940d955-21e5-41a3-91dc-4d8f0c5f4dd9 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:53.871914 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940d955-21e5-41a3-91dc-4d8f0c5f4dd9 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:53.872125 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f940d955-21e5-41a3-91dc-4d8f0c5f4dd9 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:53.873816 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 686/1372] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:59:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:59:53.880456 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0d900-e0e9-444e-8b06-6d1f0d40e581 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:53.880717 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0d900-e0e9-444e-8b06-6d1f0d40e581 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:53.880976 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec0d900-e0e9-444e-8b06-6d1f0d40e581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:53.900723 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-5ec0d900-e0e9-444e-8b06-6d1f0d40e581 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:59:53.901949 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5ec0d900-e0e9-444e-8b06-6d1f0d40e581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:53.910491 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 687/1373] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:59:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:59:54.218454 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919e6a0-b126-4f14-962f-a977fd11c5cc None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:54.218733 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919e6a0-b126-4f14-962f-a977fd11c5cc None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:54.219038 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0919e6a0-b126-4f14-962f-a977fd11c5cc None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:54.220553 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 687/1374] 10.222.0.13 () {54 vars in 963 bytes} [Mon May 23 16:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:59:54.226740 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5d4fa-9099-42e6-b5a6-98cd36c8c352 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:54.227184 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5d4fa-9099-42e6-b5a6-98cd36c8c352 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:54.228079 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed5d4fa-9099-42e6-b5a6-98cd36c8c352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:54.287909 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-3ed5d4fa-9099-42e6-b5a6-98cd36c8c352 None None] MFA Rules not processed for user `22e45ab9d34c4fcbaf93be339257eca8`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:59:54.289191 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-3ed5d4fa-9099-42e6-b5a6-98cd36c8c352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:54.353605 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 688/1375] 10.222.0.13 () {58 vars in 1060 bytes} [Mon May 23 16:59:54 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:59:54.362821 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919e6a0-b126-4f14-962f-a977fd11c5cc None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:59:54.374748 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0919e6a0-b126-4f14-962f-a977fd11c5cc None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:59:54.378680 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85f6d53d-a14c-4495-b030-c9f2207c05c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22e45ab9d34c4fcbaf93be339257eca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:59:54.379742 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6d53d-a14c-4495-b030-c9f2207c05c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:54.380103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6d53d-a14c-4495-b030-c9f2207c05c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:54.380449 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85f6d53d-a14c-4495-b030-c9f2207c05c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:54.390503 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f6d53d-a14c-4495-b030-c9f2207c05c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:59:54.392159 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85f6d53d-a14c-4495-b030-c9f2207c05c0 service cinder] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:59:54.403145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 688/1376] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 16:59:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:59:55.997218 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5afaf-2d69-4093-8355-77b095ad2d65 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:55.997708 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5afaf-2d69-4093-8355-77b095ad2d65 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:55.997806 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce5afaf-2d69-4093-8355-77b095ad2d65 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:55.999609 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 689/1377] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:59:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:59:56.005750 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97fa842-20e8-4f90-bf28-e5ce347bfbd8 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:56.006289 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97fa842-20e8-4f90-bf28-e5ce347bfbd8 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:56.006289 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97fa842-20e8-4f90-bf28-e5ce347bfbd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:56.026606 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-e97fa842-20e8-4f90-bf28-e5ce347bfbd8 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:59:56.027868 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-e97fa842-20e8-4f90-bf28-e5ce347bfbd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:56.035938 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 689/1378] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:59:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:59:56.081783 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5afaf-2d69-4093-8355-77b095ad2d65 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:59:56.097746 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce5afaf-2d69-4093-8355-77b095ad2d65 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:59:56.104641 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd6d488-0f01-4b5e-b44c-136c84b5e0c8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:59:56.105309 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6d488-0f01-4b5e-b44c-136c84b5e0c8 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:56.105588 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6d488-0f01-4b5e-b44c-136c84b5e0c8 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:56.105862 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd6d488-0f01-4b5e-b44c-136c84b5e0c8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:56.115980 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6d488-0f01-4b5e-b44c-136c84b5e0c8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:59:56.117757 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd6d488-0f01-4b5e-b44c-136c84b5e0c8 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:59:56.122097 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 16:59:56.129964 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 690/1379] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 16:59:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 16:59:58.366118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7592944-0879-4139-ab2b-69f1e3d059fe None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:58.366793 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7592944-0879-4139-ab2b-69f1e3d059fe None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:58.366793 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7592944-0879-4139-ab2b-69f1e3d059fe None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:58.368135 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 690/1380] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 16:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 16:59:58.373991 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cdbdda-f08e-48ff-838f-98546ec4e95c None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:58.374443 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cdbdda-f08e-48ff-838f-98546ec4e95c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:58.374538 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82cdbdda-f08e-48ff-838f-98546ec4e95c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:58.394775 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-82cdbdda-f08e-48ff-838f-98546ec4e95c None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 16:59:58.396049 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-82cdbdda-f08e-48ff-838f-98546ec4e95c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 16:59:58.411404 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 691/1381] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 16:59:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 16:59:58.450297 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7592944-0879-4139-ab2b-69f1e3d059fe None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 16:59:58.469590 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7592944-0879-4139-ab2b-69f1e3d059fe None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 16:59:58.475177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f371a2f5-e91e-4981-b4c2-d209e4da4e44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 16:59:58.476455 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371a2f5-e91e-4981-b4c2-d209e4da4e44 service nova] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 16:59:58.476912 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371a2f5-e91e-4981-b4c2-d209e4da4e44 service nova] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 16:59:58.477352 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f371a2f5-e91e-4981-b4c2-d209e4da4e44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 16:59:58.487341 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f371a2f5-e91e-4981-b4c2-d209e4da4e44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 16:59:58.489284 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f371a2f5-e91e-4981-b4c2-d209e4da4e44 service nova] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 16:59:58.499665 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 691/1382] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 16:59:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:00:00.723927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca5ccc-0ae0-4d1e-9cbb-47313d51cc09 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:00.724478 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca5ccc-0ae0-4d1e-9cbb-47313d51cc09 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:00.724478 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1ca5ccc-0ae0-4d1e-9cbb-47313d51cc09 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:00.725832 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 692/1383] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 17:00:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:00:00.731640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddcaca8-683d-45c6-ba57-b4a708eca31f None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:00.732062 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddcaca8-683d-45c6-ba57-b4a708eca31f None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:00.732145 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddcaca8-683d-45c6-ba57-b4a708eca31f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:00.752122 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-1ddcaca8-683d-45c6-ba57-b4a708eca31f None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:00.753402 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-1ddcaca8-683d-45c6-ba57-b4a708eca31f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:00.762282 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 692/1384] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 17:00:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:00:00.804726 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca5ccc-0ae0-4d1e-9cbb-47313d51cc09 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:00:00.820256 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1ca5ccc-0ae0-4d1e-9cbb-47313d51cc09 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:00:00.826442 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dcea3e-b4e6-41b3-ae0a-812a23d7968b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:00:00.827124 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcea3e-b4e6-41b3-ae0a-812a23d7968b service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:00.827406 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcea3e-b4e6-41b3-ae0a-812a23d7968b service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:00.827677 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dcea3e-b4e6-41b3-ae0a-812a23d7968b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:00.836817 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dcea3e-b4e6-41b3-ae0a-812a23d7968b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:00:00.838500 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dcea3e-b4e6-41b3-ae0a-812a23d7968b service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:00:00.855130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 693/1385] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:00:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:00:03.043697 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d778a58-4ce4-42eb-aa9b-54aa05278f8b None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:03.044273 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d778a58-4ce4-42eb-aa9b-54aa05278f8b None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:03.044273 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d778a58-4ce4-42eb-aa9b-54aa05278f8b None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:03.045523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 693/1386] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 17:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:00:03.051548 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231df250-fb7c-4bd4-bc5e-d4d28fa56a61 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:03.051989 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231df250-fb7c-4bd4-bc5e-d4d28fa56a61 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:03.052108 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231df250-fb7c-4bd4-bc5e-d4d28fa56a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:03.071327 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-231df250-fb7c-4bd4-bc5e-d4d28fa56a61 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:03.072529 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-231df250-fb7c-4bd4-bc5e-d4d28fa56a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:03.080505 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 694/1387] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 17:00:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:00:03.127454 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d778a58-4ce4-42eb-aa9b-54aa05278f8b None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:00:03.147330 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d778a58-4ce4-42eb-aa9b-54aa05278f8b None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:00:03.155534 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a40c8c-e080-4f48-a4fc-b6f1e29a76c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:00:03.156341 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a40c8c-e080-4f48-a4fc-b6f1e29a76c2 service nova] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:03.156884 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a40c8c-e080-4f48-a4fc-b6f1e29a76c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:03.157227 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a40c8c-e080-4f48-a4fc-b6f1e29a76c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:03.168995 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a40c8c-e080-4f48-a4fc-b6f1e29a76c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:00:03.171386 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a40c8c-e080-4f48-a4fc-b6f1e29a76c2 service nova] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:00:03.184084 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 694/1388] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:00:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:00:06.049839 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c59b1-b967-4e18-b413-b92b994b203f None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:06.050430 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c59b1-b967-4e18-b413-b92b994b203f None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:06.050697 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3c59b1-b967-4e18-b413-b92b994b203f None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:06.053137 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 695/1389] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 17:00:06 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:00:06.060292 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae366060-5ef7-4adc-a7d0-1ea47886e8ec None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:06.060794 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae366060-5ef7-4adc-a7d0-1ea47886e8ec None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:06.061067 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae366060-5ef7-4adc-a7d0-1ea47886e8ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:06.087179 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-ae366060-5ef7-4adc-a7d0-1ea47886e8ec None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:06.088985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-ae366060-5ef7-4adc-a7d0-1ea47886e8ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:06.099492 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 695/1390] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 17:00:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:00:06.146612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c59b1-b967-4e18-b413-b92b994b203f None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:00:06.161272 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3c59b1-b967-4e18-b413-b92b994b203f None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:00:06.167299 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6b46e-2d98-4f2c-8bcf-7a1c694949fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:00:06.167937 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6b46e-2d98-4f2c-8bcf-7a1c694949fa service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:06.168215 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6b46e-2d98-4f2c-8bcf-7a1c694949fa service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:06.168523 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6b46e-2d98-4f2c-8bcf-7a1c694949fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:06.177370 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6b46e-2d98-4f2c-8bcf-7a1c694949fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:00:06.179045 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6b46e-2d98-4f2c-8bcf-7a1c694949fa service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:00:06.189409 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 696/1391] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:00:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:00:08.468265 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc23543-9a0c-4c6f-a1f2-1fc397708566 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:08.468932 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc23543-9a0c-4c6f-a1f2-1fc397708566 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:08.468932 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc23543-9a0c-4c6f-a1f2-1fc397708566 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:08.470170 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 696/1392] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 17:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:00:08.475666 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06400f07-f616-46eb-b692-d8b3df46c94d None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:08.476186 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06400f07-f616-46eb-b692-d8b3df46c94d None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:08.476283 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06400f07-f616-46eb-b692-d8b3df46c94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:08.497192 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-06400f07-f616-46eb-b692-d8b3df46c94d None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:08.498384 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-06400f07-f616-46eb-b692-d8b3df46c94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:08.507099 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 697/1393] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 17:00:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:00:08.551089 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc23543-9a0c-4c6f-a1f2-1fc397708566 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:00:08.566221 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc23543-9a0c-4c6f-a1f2-1fc397708566 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:00:08.572952 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd9775b-f22b-4bcb-9ffd-20fe803548c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:00:08.573600 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9775b-f22b-4bcb-9ffd-20fe803548c4 service nova] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:08.573879 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9775b-f22b-4bcb-9ffd-20fe803548c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:08.574157 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd9775b-f22b-4bcb-9ffd-20fe803548c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:08.582677 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9775b-f22b-4bcb-9ffd-20fe803548c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:00:08.584306 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd9775b-f22b-4bcb-9ffd-20fe803548c4 service nova] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:00:08.594532 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 697/1394] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:00:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:00:10.047599 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd609e33-c80e-437c-b8a0-89f11dc5ed94 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:10.048128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd609e33-c80e-437c-b8a0-89f11dc5ed94 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:10.048128 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd609e33-c80e-437c-b8a0-89f11dc5ed94 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:10.049667 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 698/1395] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 17:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:00:10.055460 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03192dc7-7f2f-4fb2-aba9-0eb1b2fdf401 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:10.055924 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03192dc7-7f2f-4fb2-aba9-0eb1b2fdf401 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:10.056060 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03192dc7-7f2f-4fb2-aba9-0eb1b2fdf401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:10.074262 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-03192dc7-7f2f-4fb2-aba9-0eb1b2fdf401 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:10.075462 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-03192dc7-7f2f-4fb2-aba9-0eb1b2fdf401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:10.084055 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 698/1396] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 17:00:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:00:10.138798 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d28cec-b907-4a9c-8014-790b375d3277 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:10.139129 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d28cec-b907-4a9c-8014-790b375d3277 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:10.139406 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d28cec-b907-4a9c-8014-790b375d3277 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:10.140770 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 699/1397] 10.222.0.13 () {54 vars in 962 bytes} [Mon May 23 17:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:00:10.146416 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad36ff7-80cc-4e9d-8275-00d8197e41d2 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:10.146699 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad36ff7-80cc-4e9d-8275-00d8197e41d2 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:10.146999 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad36ff7-80cc-4e9d-8275-00d8197e41d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:10.197170 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-aad36ff7-80cc-4e9d-8275-00d8197e41d2 None None] MFA Rules not processed for user `d93036329cf64a8e91edaee5b1d47b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:10.198369 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-aad36ff7-80cc-4e9d-8275-00d8197e41d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:10.260668 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 699/1398] 10.222.0.13 () {58 vars in 1059 bytes} [Mon May 23 17:00:10 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:00:10.268431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d28cec-b907-4a9c-8014-790b375d3277 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:00:10.280600 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d28cec-b907-4a9c-8014-790b375d3277 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:00:10.284467 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478fcc24-0073-48ee-b723-125bda9a8770 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd93036329cf64a8e91edaee5b1d47b9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:00:10.285101 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fcc24-0073-48ee-b723-125bda9a8770 service glance] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:10.285419 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fcc24-0073-48ee-b723-125bda9a8770 service glance] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:10.285725 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478fcc24-0073-48ee-b723-125bda9a8770 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:10.295479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478fcc24-0073-48ee-b723-125bda9a8770 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:00:10.296906 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-478fcc24-0073-48ee-b723-125bda9a8770 service glance] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:00:10.307037 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 700/1399] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:00:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:00:10.808809 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f67d2-badd-4a15-9007-a1c44f4c3127 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:10.809153 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f67d2-badd-4a15-9007-a1c44f4c3127 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:10.809463 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881f67d2-badd-4a15-9007-a1c44f4c3127 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:10.810750 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 700/1400] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 17:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 17:00:10.816509 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bf187-afb8-473a-a1b6-fe323703a464 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:10.816791 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bf187-afb8-473a-a1b6-fe323703a464 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:10.817067 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6bf187-afb8-473a-a1b6-fe323703a464 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:10.836577 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9f7a7-966d-4973-ab60-8c126ea55d0d None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:10.836744 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-db6bf187-afb8-473a-a1b6-fe323703a464 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:10.836885 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9f7a7-966d-4973-ab60-8c126ea55d0d None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:10.837162 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e9f7a7-966d-4973-ab60-8c126ea55d0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:10.837899 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-db6bf187-afb8-473a-a1b6-fe323703a464 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:10.842555 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=93 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 17:00:10.845826 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 701/1401] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 17:00:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 17:00:10.887103 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-d5e9f7a7-966d-4973-ab60-8c126ea55d0d None None] MFA Rules not processed for user `4b2d638e6bd240ccbe45228352900bdb`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:10.888449 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-d5e9f7a7-966d-4973-ab60-8c126ea55d0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:10.889138 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6bf187-afb8-473a-a1b6-fe323703a464 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:00:10.901520 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6bf187-afb8-473a-a1b6-fe323703a464 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:00:10.906793 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bcdf4f-dead-47bd-a518-69c821c38bc4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:00:10.907461 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bcdf4f-dead-47bd-a518-69c821c38bc4 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:10.907731 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bcdf4f-dead-47bd-a518-69c821c38bc4 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:10.908001 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bcdf4f-dead-47bd-a518-69c821c38bc4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:10.917352 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bcdf4f-dead-47bd-a518-69c821c38bc4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:00:10.918944 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bcdf4f-dead-47bd-a518-69c821c38bc4 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:00:10.928454 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 702/1402] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:00:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 17:00:10.955058 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 701/1403] 10.222.0.13 () {58 vars in 1009 bytes} [Mon May 23 17:00:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 17:00:10.998153 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57bb28-5175-42c8-a8f3-6d2c8f2dc20c None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:10.998456 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57bb28-5175-42c8-a8f3-6d2c8f2dc20c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:10.998734 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be57bb28-5175-42c8-a8f3-6d2c8f2dc20c None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:11.000174 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 703/1404] 10.222.0.13 () {54 vars in 942 bytes} [Mon May 23 17:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:00:11.006894 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533590a-dab2-4cfe-9be5-75b1b46e155d None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:11.007356 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533590a-dab2-4cfe-9be5-75b1b46e155d None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:11.007805 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9533590a-dab2-4cfe-9be5-75b1b46e155d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:11.066546 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-9533590a-dab2-4cfe-9be5-75b1b46e155d None None] MFA Rules not processed for user `6546217667054abb8eb07076995c1b04`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:11.067752 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-9533590a-dab2-4cfe-9be5-75b1b46e155d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:11.151395 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 702/1405] 10.222.0.13 () {58 vars in 1039 bytes} [Mon May 23 17:00:11 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:00:11.161536 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57bb28-5175-42c8-a8f3-6d2c8f2dc20c None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:00:11.177172 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be57bb28-5175-42c8-a8f3-6d2c8f2dc20c None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:00:11.183262 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1610e4eb-c98a-4006-bc1e-8d6a578ca5b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6546217667054abb8eb07076995c1b04', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:00:11.183936 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610e4eb-c98a-4006-bc1e-8d6a578ca5b7 service swift] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:11.184221 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610e4eb-c98a-4006-bc1e-8d6a578ca5b7 service swift] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:11.184495 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1610e4eb-c98a-4006-bc1e-8d6a578ca5b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:11.194795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610e4eb-c98a-4006-bc1e-8d6a578ca5b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:00:11.196747 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1610e4eb-c98a-4006-bc1e-8d6a578ca5b7 service swift] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:00:11.205181 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 704/1406] 10.222.0.13 () {60 vars in 1384 bytes} [Mon May 23 17:00:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 17:00:13.253522 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150cc31-4905-4236-9382-9c8c14abe108 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:13.254854 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150cc31-4905-4236-9382-9c8c14abe108 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:13.255395 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7150cc31-4905-4236-9382-9c8c14abe108 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:13.261184 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 703/1407] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 17:00:13 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:00:13.262589 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98edf5-7364-48ce-93c3-26836cca373c None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:13.263022 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98edf5-7364-48ce-93c3-26836cca373c None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:13.263208 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c98edf5-7364-48ce-93c3-26836cca373c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:13.283595 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-7c98edf5-7364-48ce-93c3-26836cca373c None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:13.285259 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7c98edf5-7364-48ce-93c3-26836cca373c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:13.298960 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 705/1408] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 17:00:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:00:13.341166 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150cc31-4905-4236-9382-9c8c14abe108 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:00:13.359785 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7150cc31-4905-4236-9382-9c8c14abe108 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:00:13.370490 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a339309a-de6d-4673-bfb3-03bba0573434 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:00:13.371269 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339309a-de6d-4673-bfb3-03bba0573434 service nova] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:13.371497 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339309a-de6d-4673-bfb3-03bba0573434 service nova] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:13.371776 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a339309a-de6d-4673-bfb3-03bba0573434 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:13.381358 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339309a-de6d-4673-bfb3-03bba0573434 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:00:13.384144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a339309a-de6d-4673-bfb3-03bba0573434 service nova] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:00:13.408060 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 704/1409] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:00:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:00:23.553488 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=46 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 17:00:23.882584 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=89 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 17:00:34.888612 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cecec5-5251-4fb1-a0b1-e27764761089 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:34.889239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cecec5-5251-4fb1-a0b1-e27764761089 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:34.889239 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cecec5-5251-4fb1-a0b1-e27764761089 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:34.890545 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 706/1410] 10.222.0.13 () {54 vars in 918 bytes} [Mon May 23 17:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:00:34.896085 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437993dc-b779-434d-a223-16514a23b8e6 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:34.896570 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437993dc-b779-434d-a223-16514a23b8e6 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:34.896675 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437993dc-b779-434d-a223-16514a23b8e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:34.917604 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-437993dc-b779-434d-a223-16514a23b8e6 None None] MFA Rules not processed for user `f181ebdfcd544b54af7e6f9c118fe252`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:34.918783 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-437993dc-b779-434d-a223-16514a23b8e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:34.928591 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 705/1411] 10.222.0.13 () {58 vars in 1015 bytes} [Mon May 23 17:00:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:00:34.943344 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a21bc8-f58e-4f1d-bbff-462102ed1ffd None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:34.943630 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a21bc8-f58e-4f1d-bbff-462102ed1ffd None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:34.943904 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a21bc8-f58e-4f1d-bbff-462102ed1ffd None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:34.945382 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 707/1412] 10.222.0.13 () {54 vars in 918 bytes} [Mon May 23 17:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:00:34.950116 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044635bf-242b-481c-b67a-905bae28b006 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:34.950397 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044635bf-242b-481c-b67a-905bae28b006 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:34.950673 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044635bf-242b-481c-b67a-905bae28b006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:34.969382 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-044635bf-242b-481c-b67a-905bae28b006 None None] MFA Rules not processed for user `225c7b385f12445fab86b3b5606b1da1`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:34.970537 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-044635bf-242b-481c-b67a-905bae28b006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:34.980516 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 706/1413] 10.222.0.13 () {58 vars in 1015 bytes} [Mon May 23 17:00:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:00:37.718842 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6bb33-c8b9-40e7-9c70-d8a6b2fcce7b None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:37.719790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6bb33-c8b9-40e7-9c70-d8a6b2fcce7b None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:37.719790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6bb33-c8b9-40e7-9c70-d8a6b2fcce7b None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:37.721943 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 708/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 17:00:37 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:00:37.729277 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0c586-11f1-46e4-a9ea-bcfedbf1dbcc None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:37.729755 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0c586-11f1-46e4-a9ea-bcfedbf1dbcc None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:37.729755 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e0c586-11f1-46e4-a9ea-bcfedbf1dbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:37.748038 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-a9e0c586-11f1-46e4-a9ea-bcfedbf1dbcc None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:37.749204 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-a9e0c586-11f1-46e4-a9ea-bcfedbf1dbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:37.757803 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 707/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 17:00:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:00:37.805043 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6bb33-c8b9-40e7-9c70-d8a6b2fcce7b None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:00:37.822340 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6bb33-c8b9-40e7-9c70-d8a6b2fcce7b None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:00:37.827990 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae06c534-1586-470f-90c3-798c022d3402 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:00:37.828624 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06c534-1586-470f-90c3-798c022d3402 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:37.828896 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06c534-1586-470f-90c3-798c022d3402 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:37.829164 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae06c534-1586-470f-90c3-798c022d3402 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:37.837458 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06c534-1586-470f-90c3-798c022d3402 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:00:37.839070 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae06c534-1586-470f-90c3-798c022d3402 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:00:37.848445 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 709/1416] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:00:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:00:38.826203 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63a073-b032-4580-8661-a12bb25439f0 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:38.826784 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63a073-b032-4580-8661-a12bb25439f0 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:38.826784 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b63a073-b032-4580-8661-a12bb25439f0 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:38.828053 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 708/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 17:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:00:38.833172 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77604355-6cce-44f1-b419-d80764742cc8 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:38.833626 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77604355-6cce-44f1-b419-d80764742cc8 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:38.833732 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77604355-6cce-44f1-b419-d80764742cc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:38.853431 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-77604355-6cce-44f1-b419-d80764742cc8 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:38.854609 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-77604355-6cce-44f1-b419-d80764742cc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:38.863883 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 710/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 17:00:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:00:39.017713 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63a073-b032-4580-8661-a12bb25439f0 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:00:39.034533 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b63a073-b032-4580-8661-a12bb25439f0 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:00:39.040586 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccee9893-b9bc-4958-aad5-ce0f8997ae26 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:00:39.041220 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee9893-b9bc-4958-aad5-ce0f8997ae26 service nova] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:39.041504 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee9893-b9bc-4958-aad5-ce0f8997ae26 service nova] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:39.041782 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccee9893-b9bc-4958-aad5-ce0f8997ae26 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:39.050402 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee9893-b9bc-4958-aad5-ce0f8997ae26 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:00:39.052042 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccee9893-b9bc-4958-aad5-ce0f8997ae26 service nova] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:00:39.061589 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 709/1419] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:00:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:00:39.993138 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a086817-5423-4f9c-a1db-adb3c14fa12b None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:39.993785 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a086817-5423-4f9c-a1db-adb3c14fa12b None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:39.993785 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a086817-5423-4f9c-a1db-adb3c14fa12b None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:39.995115 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 711/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 17:00:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:00:40.001443 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6b727-a319-4c1e-881c-2e4bdcaa24c3 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:40.002160 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6b727-a319-4c1e-881c-2e4bdcaa24c3 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:40.002348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d6b727-a319-4c1e-881c-2e4bdcaa24c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:40.023829 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-e7d6b727-a319-4c1e-881c-2e4bdcaa24c3 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:40.025031 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-e7d6b727-a319-4c1e-881c-2e4bdcaa24c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:40.033271 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 710/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 17:00:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:00:40.197646 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a086817-5423-4f9c-a1db-adb3c14fa12b None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:00:40.217462 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a086817-5423-4f9c-a1db-adb3c14fa12b None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:00:40.223026 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7c4be1-392d-463c-9a05-b8593ab480e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:00:40.223669 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c4be1-392d-463c-9a05-b8593ab480e6 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:40.223950 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c4be1-392d-463c-9a05-b8593ab480e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:40.224218 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7c4be1-392d-463c-9a05-b8593ab480e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:40.232478 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c4be1-392d-463c-9a05-b8593ab480e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:00:40.234074 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7c4be1-392d-463c-9a05-b8593ab480e6 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:00:40.243785 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 712/1422] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:00:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:00:41.146200 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4197cca-4956-439b-97ca-aec36c12906e None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:41.147104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4197cca-4956-439b-97ca-aec36c12906e None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:41.147199 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4197cca-4956-439b-97ca-aec36c12906e None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:41.149262 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 711/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 17:00:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:00:41.155554 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d01da6-587e-4280-83c5-3faec3c748c8 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:41.155957 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d01da6-587e-4280-83c5-3faec3c748c8 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:41.156112 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d01da6-587e-4280-83c5-3faec3c748c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:41.183510 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-90d01da6-587e-4280-83c5-3faec3c748c8 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:41.184716 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-90d01da6-587e-4280-83c5-3faec3c748c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:41.194234 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 713/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 17:00:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:00:41.348707 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4197cca-4956-439b-97ca-aec36c12906e None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:00:41.363783 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4197cca-4956-439b-97ca-aec36c12906e None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:00:41.370547 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b53e7f6-29b9-445c-82cd-863f1c91895c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:00:41.371228 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53e7f6-29b9-445c-82cd-863f1c91895c service nova] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:41.371511 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53e7f6-29b9-445c-82cd-863f1c91895c service nova] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:41.371814 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b53e7f6-29b9-445c-82cd-863f1c91895c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:41.380981 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53e7f6-29b9-445c-82cd-863f1c91895c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:00:41.383095 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b53e7f6-29b9-445c-82cd-863f1c91895c service nova] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:00:41.395625 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 712/1425] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:00:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:00:42.301926 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332deab-ea8c-4356-a4a3-eaa34c6c3307 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:42.302795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332deab-ea8c-4356-a4a3-eaa34c6c3307 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:42.302795 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332deab-ea8c-4356-a4a3-eaa34c6c3307 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:42.305615 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 714/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 17:00:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:00:42.313291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f502d-d7f9-4e98-a0da-a34b89e56dd6 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:42.313719 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f502d-d7f9-4e98-a0da-a34b89e56dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:42.313834 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5f502d-d7f9-4e98-a0da-a34b89e56dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:42.335386 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-4a5f502d-d7f9-4e98-a0da-a34b89e56dd6 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:42.336725 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-4a5f502d-d7f9-4e98-a0da-a34b89e56dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:42.345822 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 713/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 17:00:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:00:42.494990 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332deab-ea8c-4356-a4a3-eaa34c6c3307 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:00:42.519607 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332deab-ea8c-4356-a4a3-eaa34c6c3307 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:00:42.526904 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e3ab33-67fd-401f-9825-77a06f51fac2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:00:42.527611 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3ab33-67fd-401f-9825-77a06f51fac2 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:42.527893 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3ab33-67fd-401f-9825-77a06f51fac2 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:42.528169 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e3ab33-67fd-401f-9825-77a06f51fac2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:42.537288 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3ab33-67fd-401f-9825-77a06f51fac2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:00:42.539081 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e3ab33-67fd-401f-9825-77a06f51fac2 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:00:42.548755 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 715/1428] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:00:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:00:43.412841 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596eda9-291b-4132-8225-08e0f2cce5cf None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:43.413365 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596eda9-291b-4132-8225-08e0f2cce5cf None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:43.413462 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d596eda9-291b-4132-8225-08e0f2cce5cf None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:43.414944 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 714/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 17:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:00:43.420537 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331a3a8-03b9-4986-bb0a-cbd0f94ed28b None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:43.420987 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331a3a8-03b9-4986-bb0a-cbd0f94ed28b None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:43.421098 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1331a3a8-03b9-4986-bb0a-cbd0f94ed28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:43.440402 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-1331a3a8-03b9-4986-bb0a-cbd0f94ed28b None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:43.441589 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-1331a3a8-03b9-4986-bb0a-cbd0f94ed28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:43.450948 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 716/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 17:00:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 17:00:43.600158 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596eda9-291b-4132-8225-08e0f2cce5cf None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:00:43.616813 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d596eda9-291b-4132-8225-08e0f2cce5cf None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:00:43.624147 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace22f64-f9b1-4026-89e1-e4b7d8168078 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:00:43.624785 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace22f64-f9b1-4026-89e1-e4b7d8168078 service nova] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:43.625062 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace22f64-f9b1-4026-89e1-e4b7d8168078 service nova] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:43.625329 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace22f64-f9b1-4026-89e1-e4b7d8168078 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:43.637840 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace22f64-f9b1-4026-89e1-e4b7d8168078 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:00:43.641727 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace22f64-f9b1-4026-89e1-e4b7d8168078 service nova] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:00:43.663600 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 715/1431] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:00:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:00:44.603286 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed76297-61a4-407e-85a9-5b266b32f2e5 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:44.604201 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed76297-61a4-407e-85a9-5b266b32f2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:44.604201 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed76297-61a4-407e-85a9-5b266b32f2e5 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:44.606501 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 717/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 17:00:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:00:44.614716 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39cacd3-1451-40ea-b28d-f1444e49c6f0 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:44.615232 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39cacd3-1451-40ea-b28d-f1444e49c6f0 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:44.615355 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39cacd3-1451-40ea-b28d-f1444e49c6f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:44.641282 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-b39cacd3-1451-40ea-b28d-f1444e49c6f0 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:44.642509 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-b39cacd3-1451-40ea-b28d-f1444e49c6f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:44.652072 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 716/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 17:00:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:00:44.802564 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed76297-61a4-407e-85a9-5b266b32f2e5 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:00:44.819280 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed76297-61a4-407e-85a9-5b266b32f2e5 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:00:44.828542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c4dadb-c2ff-4958-8b41-b1ca5cf37ddb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:00:44.829583 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4dadb-c2ff-4958-8b41-b1ca5cf37ddb service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:44.830044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4dadb-c2ff-4958-8b41-b1ca5cf37ddb service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:44.830445 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c4dadb-c2ff-4958-8b41-b1ca5cf37ddb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:44.842245 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4dadb-c2ff-4958-8b41-b1ca5cf37ddb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:00:44.843880 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c4dadb-c2ff-4958-8b41-b1ca5cf37ddb service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:00:44.854062 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 718/1434] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:00:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:00:45.707719 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c032b2-742f-4625-887e-b3adad2de400 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:45.708253 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c032b2-742f-4625-887e-b3adad2de400 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:45.708253 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c032b2-742f-4625-887e-b3adad2de400 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:45.709546 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 717/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 17:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:00:45.715631 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f30d7c-86a8-4b54-acd4-c9a5f046f48e None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:45.716074 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f30d7c-86a8-4b54-acd4-c9a5f046f48e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:45.716317 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f30d7c-86a8-4b54-acd4-c9a5f046f48e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:45.742868 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-70f30d7c-86a8-4b54-acd4-c9a5f046f48e None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:45.744141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-70f30d7c-86a8-4b54-acd4-c9a5f046f48e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:45.752444 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 719/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 17:00:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:00:45.903828 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c032b2-742f-4625-887e-b3adad2de400 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:00:45.924133 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c032b2-742f-4625-887e-b3adad2de400 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:00:45.930951 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df533516-186f-4b55-af34-08597e0395b8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:00:45.931639 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df533516-186f-4b55-af34-08597e0395b8 service nova] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:45.931919 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df533516-186f-4b55-af34-08597e0395b8 service nova] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:45.932190 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df533516-186f-4b55-af34-08597e0395b8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:45.941859 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df533516-186f-4b55-af34-08597e0395b8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:00:45.943471 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df533516-186f-4b55-af34-08597e0395b8 service nova] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:00:45.953469 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 718/1437] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:00:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:00:46.832441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cdaaa9-2a3e-492f-b56c-19b9a5c91058 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:46.833327 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cdaaa9-2a3e-492f-b56c-19b9a5c91058 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:46.833327 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4cdaaa9-2a3e-492f-b56c-19b9a5c91058 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:46.835635 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 720/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 17:00:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:00:46.843972 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c815ac-2bcb-403e-ac17-c9830489dc80 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:46.844427 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c815ac-2bcb-403e-ac17-c9830489dc80 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:46.844529 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c815ac-2bcb-403e-ac17-c9830489dc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:46.868071 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-a1c815ac-2bcb-403e-ac17-c9830489dc80 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:46.869787 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-a1c815ac-2bcb-403e-ac17-c9830489dc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:46.883523 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 719/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 17:00:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:00:47.029621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cdaaa9-2a3e-492f-b56c-19b9a5c91058 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:00:47.054527 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4cdaaa9-2a3e-492f-b56c-19b9a5c91058 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:00:47.061943 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a63be2-3f5b-4994-9dae-1b0b36f7a23a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:00:47.062615 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a63be2-3f5b-4994-9dae-1b0b36f7a23a service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:47.062899 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a63be2-3f5b-4994-9dae-1b0b36f7a23a service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:47.063324 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a63be2-3f5b-4994-9dae-1b0b36f7a23a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:47.072827 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a63be2-3f5b-4994-9dae-1b0b36f7a23a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:00:47.074437 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a63be2-3f5b-4994-9dae-1b0b36f7a23a service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:00:47.084665 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 721/1440] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:00:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:00:49.149936 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517e2e5-faa5-48b5-a70f-dfa238f10884 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:49.150812 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517e2e5-faa5-48b5-a70f-dfa238f10884 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:49.150812 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8517e2e5-faa5-48b5-a70f-dfa238f10884 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:49.153113 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 720/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 17:00:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:00:49.159552 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f0cc7-220a-464c-89e2-c1c7f7f461de None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:49.160332 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f0cc7-220a-464c-89e2-c1c7f7f461de None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:49.160488 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1f0cc7-220a-464c-89e2-c1c7f7f461de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:49.180044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-ed1f0cc7-220a-464c-89e2-c1c7f7f461de None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:49.181235 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-ed1f0cc7-220a-464c-89e2-c1c7f7f461de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:49.189919 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 722/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 17:00:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:00:49.231361 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517e2e5-faa5-48b5-a70f-dfa238f10884 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:00:49.248167 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8517e2e5-faa5-48b5-a70f-dfa238f10884 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:00:49.252736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ceb4857-3f4d-4add-955b-3f5506f866ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:00:49.253370 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb4857-3f4d-4add-955b-3f5506f866ed service neutron] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:49.253639 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb4857-3f4d-4add-955b-3f5506f866ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:49.253909 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ceb4857-3f4d-4add-955b-3f5506f866ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:49.264779 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb4857-3f4d-4add-955b-3f5506f866ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:00:49.266205 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ceb4857-3f4d-4add-955b-3f5506f866ed service neutron] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:00:49.276200 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 721/1443] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:00:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:00:51.737662 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed597ca-d05d-42fc-9a7c-7ec248435ff6 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:51.738479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed597ca-d05d-42fc-9a7c-7ec248435ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:51.738479 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed597ca-d05d-42fc-9a7c-7ec248435ff6 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:51.741574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 723/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 17:00:51 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:00:51.747367 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86aa284-30af-4e14-8a59-285bfcc42797 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:51.747812 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86aa284-30af-4e14-8a59-285bfcc42797 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:51.747909 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86aa284-30af-4e14-8a59-285bfcc42797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:51.772585 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-a86aa284-30af-4e14-8a59-285bfcc42797 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:51.773820 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-a86aa284-30af-4e14-8a59-285bfcc42797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:51.783068 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 722/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 17:00:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:00:51.831364 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed597ca-d05d-42fc-9a7c-7ec248435ff6 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:00:51.851124 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed597ca-d05d-42fc-9a7c-7ec248435ff6 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:00:51.855732 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a05fb94-5c6a-49e6-986a-323b314cceec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:00:51.856378 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05fb94-5c6a-49e6-986a-323b314cceec service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:51.856659 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05fb94-5c6a-49e6-986a-323b314cceec service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:51.856926 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a05fb94-5c6a-49e6-986a-323b314cceec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:51.867209 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a05fb94-5c6a-49e6-986a-323b314cceec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:00:51.868649 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a05fb94-5c6a-49e6-986a-323b314cceec service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:00:51.878370 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 724/1446] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:00:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:00:54.409935 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5412a80-3fe2-4316-8a68-0c4b05b11fbd None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:54.410877 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5412a80-3fe2-4316-8a68-0c4b05b11fbd None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:54.410877 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5412a80-3fe2-4316-8a68-0c4b05b11fbd None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:54.413007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 723/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 17:00:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:00:54.420675 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa3922-6cc9-47b9-bf39-552aa3ed31ce None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:54.421154 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa3922-6cc9-47b9-bf39-552aa3ed31ce None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:54.421262 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aaa3922-6cc9-47b9-bf39-552aa3ed31ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:54.440092 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-9aaa3922-6cc9-47b9-bf39-552aa3ed31ce None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:54.441264 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9aaa3922-6cc9-47b9-bf39-552aa3ed31ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:54.449493 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 725/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 17:00:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 17:00:54.494520 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5412a80-3fe2-4316-8a68-0c4b05b11fbd None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:00:54.507416 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5412a80-3fe2-4316-8a68-0c4b05b11fbd None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:00:54.511192 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5e71bd-762b-4c76-84c6-e35967a84da7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:00:54.511824 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e71bd-762b-4c76-84c6-e35967a84da7 service neutron] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:54.512097 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e71bd-762b-4c76-84c6-e35967a84da7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:54.512368 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5e71bd-762b-4c76-84c6-e35967a84da7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:54.521296 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e71bd-762b-4c76-84c6-e35967a84da7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:00:54.522941 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5e71bd-762b-4c76-84c6-e35967a84da7 service neutron] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:00:54.532609 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 724/1449] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:00:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:00:56.877518 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80304062-660a-4dec-b773-9c6ce177f61e None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:56.878119 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80304062-660a-4dec-b773-9c6ce177f61e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:56.878119 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80304062-660a-4dec-b773-9c6ce177f61e None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:56.879474 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 726/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 17:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:00:56.886401 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e243f-54d1-4663-9305-2046f076b8cf None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:56.887258 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e243f-54d1-4663-9305-2046f076b8cf None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:56.887258 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09e243f-54d1-4663-9305-2046f076b8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:56.909548 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-f09e243f-54d1-4663-9305-2046f076b8cf None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:56.910734 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-f09e243f-54d1-4663-9305-2046f076b8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:56.921420 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 725/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 17:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:00:56.962491 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80304062-660a-4dec-b773-9c6ce177f61e None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:00:56.975574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80304062-660a-4dec-b773-9c6ce177f61e None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:00:56.980427 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87b0d3-1a0f-4dd4-a73e-bfbe1a235945 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:00:56.981102 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87b0d3-1a0f-4dd4-a73e-bfbe1a235945 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:56.981314 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87b0d3-1a0f-4dd4-a73e-bfbe1a235945 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:56.981578 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87b0d3-1a0f-4dd4-a73e-bfbe1a235945 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:56.992575 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c87b0d3-1a0f-4dd4-a73e-bfbe1a235945 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:00:56.994051 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c87b0d3-1a0f-4dd4-a73e-bfbe1a235945 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:00:57.003724 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 727/1452] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:00:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:00:59.269776 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf07949-0491-4f6c-b3a9-4d6784ed3644 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:59.270647 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf07949-0491-4f6c-b3a9-4d6784ed3644 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:59.270647 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf07949-0491-4f6c-b3a9-4d6784ed3644 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:59.272790 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 726/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 17:00:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:00:59.278503 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11045dab-eeea-4639-a2f8-307df55830dd None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:59.278998 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11045dab-eeea-4639-a2f8-307df55830dd None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:59.279109 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11045dab-eeea-4639-a2f8-307df55830dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:59.297643 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-11045dab-eeea-4639-a2f8-307df55830dd None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:00:59.298823 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-11045dab-eeea-4639-a2f8-307df55830dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:00:59.307325 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 728/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 17:00:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:00:59.348354 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf07949-0491-4f6c-b3a9-4d6784ed3644 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:00:59.361217 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf07949-0491-4f6c-b3a9-4d6784ed3644 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:00:59.365601 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6caca364-55fe-476d-86a5-305ff8e618fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:00:59.366288 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caca364-55fe-476d-86a5-305ff8e618fe service neutron] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:00:59.366554 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caca364-55fe-476d-86a5-305ff8e618fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:00:59.366826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6caca364-55fe-476d-86a5-305ff8e618fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:00:59.376643 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caca364-55fe-476d-86a5-305ff8e618fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:00:59.378097 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6caca364-55fe-476d-86a5-305ff8e618fe service neutron] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:00:59.388154 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 727/1455] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:00:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:01:01.656557 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6be4e2-4d3c-4ed0-9a9a-33c67e33ad28 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:01.657313 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6be4e2-4d3c-4ed0-9a9a-33c67e33ad28 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:01.657313 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e6be4e2-4d3c-4ed0-9a9a-33c67e33ad28 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:01.658699 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 729/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 17:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:01:01.664747 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2c849-d9bd-4c51-83a5-793f37e642e2 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:01.665214 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2c849-d9bd-4c51-83a5-793f37e642e2 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:01.665342 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f2c849-d9bd-4c51-83a5-793f37e642e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:01.682949 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-c1f2c849-d9bd-4c51-83a5-793f37e642e2 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:01:01.684137 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-c1f2c849-d9bd-4c51-83a5-793f37e642e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:01:01.692033 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 728/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 17:01:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:01:01.745004 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6be4e2-4d3c-4ed0-9a9a-33c67e33ad28 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:01:01.759854 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e6be4e2-4d3c-4ed0-9a9a-33c67e33ad28 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:01:01.764217 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943a9de3-7a7e-4dad-82e5-5fdbc5e77460 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:01:01.764882 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a9de3-7a7e-4dad-82e5-5fdbc5e77460 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:01.765154 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a9de3-7a7e-4dad-82e5-5fdbc5e77460 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:01.765418 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943a9de3-7a7e-4dad-82e5-5fdbc5e77460 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:01.775076 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a9de3-7a7e-4dad-82e5-5fdbc5e77460 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:01:01.776515 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943a9de3-7a7e-4dad-82e5-5fdbc5e77460 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:01:01.785781 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 730/1458] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:01:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:01:04.055666 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085310d-7bf2-4c20-b074-de8839cc048a None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:04.056268 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085310d-7bf2-4c20-b074-de8839cc048a None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:04.056268 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d085310d-7bf2-4c20-b074-de8839cc048a None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:04.057564 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 729/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 17:01:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:01:04.063101 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e88066-816c-4cf9-accc-306845939326 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:04.063555 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e88066-816c-4cf9-accc-306845939326 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:04.063653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e88066-816c-4cf9-accc-306845939326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:04.072001 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=70 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 17:01:04.085695 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-17e88066-816c-4cf9-accc-306845939326 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:01:04.086927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-17e88066-816c-4cf9-accc-306845939326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:01:04.095393 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 731/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 17:01:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:01:04.134262 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d085310d-7bf2-4c20-b074-de8839cc048a None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:01:04.140369 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=73 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 17:01:04.147888 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d085310d-7bf2-4c20-b074-de8839cc048a None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:01:04.151848 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed724af0-dd8c-475e-bad8-af3d5c85316f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:01:04.152483 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed724af0-dd8c-475e-bad8-af3d5c85316f service neutron] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:04.152759 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed724af0-dd8c-475e-bad8-af3d5c85316f service neutron] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:04.153028 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed724af0-dd8c-475e-bad8-af3d5c85316f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:04.163423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed724af0-dd8c-475e-bad8-af3d5c85316f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:01:04.164975 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed724af0-dd8c-475e-bad8-af3d5c85316f service neutron] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:01:04.174258 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 730/1461] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:01:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:01:06.396446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841219ff-6821-4b8f-b98a-e396899c0659 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:06.397388 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841219ff-6821-4b8f-b98a-e396899c0659 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:06.397388 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841219ff-6821-4b8f-b98a-e396899c0659 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:06.399790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 732/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 17:01:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:01:06.408979 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb9588-9ed8-4806-b28f-c5006cf0059e None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:06.409731 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb9588-9ed8-4806-b28f-c5006cf0059e None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:06.409917 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb9588-9ed8-4806-b28f-c5006cf0059e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:06.430836 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-edcb9588-9ed8-4806-b28f-c5006cf0059e None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:01:06.432061 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-edcb9588-9ed8-4806-b28f-c5006cf0059e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:01:06.441440 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 731/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 17:01:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:01:06.484415 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841219ff-6821-4b8f-b98a-e396899c0659 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:01:06.498021 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841219ff-6821-4b8f-b98a-e396899c0659 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:01:06.502362 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa0e41-0d32-4b8c-97e4-749a6a255e2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:01:06.503041 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa0e41-0d32-4b8c-97e4-749a6a255e2a service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:06.503319 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa0e41-0d32-4b8c-97e4-749a6a255e2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:06.503590 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa0e41-0d32-4b8c-97e4-749a6a255e2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:06.514017 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa0e41-0d32-4b8c-97e4-749a6a255e2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:01:06.515707 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa0e41-0d32-4b8c-97e4-749a6a255e2a service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:01:06.525556 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 733/1464] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:01:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:01:08.722298 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bf905-2396-4fc0-bb4a-36bb98bdbd5a None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:08.722883 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bf905-2396-4fc0-bb4a-36bb98bdbd5a None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:08.722883 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380bf905-2396-4fc0-bb4a-36bb98bdbd5a None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:08.724973 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 732/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 17:01:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:01:08.731019 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a084d3f-6862-4cee-87c3-d014ea2c3a41 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:08.731534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a084d3f-6862-4cee-87c3-d014ea2c3a41 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:08.731534 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a084d3f-6862-4cee-87c3-d014ea2c3a41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:08.752246 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-5a084d3f-6862-4cee-87c3-d014ea2c3a41 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:01:08.753458 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-5a084d3f-6862-4cee-87c3-d014ea2c3a41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:01:08.763070 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 734/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 17:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 17:01:08.799354 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380bf905-2396-4fc0-bb4a-36bb98bdbd5a None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:01:08.810500 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380bf905-2396-4fc0-bb4a-36bb98bdbd5a None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:01:08.814127 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3329de-7df4-44d2-a4ea-6f59b95b7ddd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:01:08.814777 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3329de-7df4-44d2-a4ea-6f59b95b7ddd service neutron] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:08.815112 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3329de-7df4-44d2-a4ea-6f59b95b7ddd service neutron] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:08.815387 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3329de-7df4-44d2-a4ea-6f59b95b7ddd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:08.824528 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3329de-7df4-44d2-a4ea-6f59b95b7ddd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:01:08.825972 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3329de-7df4-44d2-a4ea-6f59b95b7ddd service neutron] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:01:08.835710 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 733/1467] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:01:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:01:11.045352 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904b92f-6d53-41cc-ac08-23707eb9ff59 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:11.046292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904b92f-6d53-41cc-ac08-23707eb9ff59 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:11.046292 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8904b92f-6d53-41cc-ac08-23707eb9ff59 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:11.048519 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 735/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 17:01:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:01:11.054308 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7292afb-9703-427b-99b8-d75b5d600696 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:11.054783 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7292afb-9703-427b-99b8-d75b5d600696 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:11.054887 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7292afb-9703-427b-99b8-d75b5d600696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:11.074167 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-f7292afb-9703-427b-99b8-d75b5d600696 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:01:11.075410 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-f7292afb-9703-427b-99b8-d75b5d600696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:01:11.084083 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 734/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 17:01:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:01:11.123546 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904b92f-6d53-41cc-ac08-23707eb9ff59 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:01:11.141361 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8904b92f-6d53-41cc-ac08-23707eb9ff59 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:01:11.147159 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc6bcc-1cdb-4e43-a2ff-0e342952882e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:01:11.147924 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc6bcc-1cdb-4e43-a2ff-0e342952882e service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:11.148205 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc6bcc-1cdb-4e43-a2ff-0e342952882e service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:11.148475 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc6bcc-1cdb-4e43-a2ff-0e342952882e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:11.159007 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc6bcc-1cdb-4e43-a2ff-0e342952882e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:01:11.160445 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7fc6bcc-1cdb-4e43-a2ff-0e342952882e service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:01:11.170821 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 736/1470] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:01:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:01:21.068492 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=17 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 17:01:21.165927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=17 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 17:01:43.981133 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca93f3e-5362-4228-b0b0-2923ead23d5c None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:43.982108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca93f3e-5362-4228-b0b0-2923ead23d5c None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:43.982108 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca93f3e-5362-4228-b0b0-2923ead23d5c None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:43.984251 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 735/1471] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 17:01:43 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:01:43.992375 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d2825-d705-44d8-a7f4-ce8dc8dfc489 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:43.992860 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d2825-d705-44d8-a7f4-ce8dc8dfc489 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:43.992979 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105d2825-d705-44d8-a7f4-ce8dc8dfc489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:44.013296 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-105d2825-d705-44d8-a7f4-ce8dc8dfc489 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:01:44.014509 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-105d2825-d705-44d8-a7f4-ce8dc8dfc489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:01:44.024224 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 737/1472] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 17:01:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:01:44.079229 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d023c2f-52c6-4001-973a-9798aa505648 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:44.079516 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d023c2f-52c6-4001-973a-9798aa505648 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:44.079789 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d023c2f-52c6-4001-973a-9798aa505648 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:44.081434 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 736/1473] 10.222.0.13 () {54 vars in 962 bytes} [Mon May 23 17:01:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:01:44.086955 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6b5b0-a839-4371-b61b-6ddd322e8a6e None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:44.087255 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6b5b0-a839-4371-b61b-6ddd322e8a6e None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:44.087536 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa6b5b0-a839-4371-b61b-6ddd322e8a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:44.106029 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-8fa6b5b0-a839-4371-b61b-6ddd322e8a6e None None] MFA Rules not processed for user `d93036329cf64a8e91edaee5b1d47b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:01:44.107249 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8fa6b5b0-a839-4371-b61b-6ddd322e8a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:01:44.117812 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 738/1474] 10.222.0.13 () {58 vars in 1059 bytes} [Mon May 23 17:01:44 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 17:01:44.125068 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d023c2f-52c6-4001-973a-9798aa505648 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:01:44.137896 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d023c2f-52c6-4001-973a-9798aa505648 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:01:44.142190 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fede04b4-da31-40f7-949e-8d38b0b700fb service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd93036329cf64a8e91edaee5b1d47b9c', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:01:44.142856 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede04b4-da31-40f7-949e-8d38b0b700fb service glance] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:44.143181 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede04b4-da31-40f7-949e-8d38b0b700fb service glance] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:44.143446 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fede04b4-da31-40f7-949e-8d38b0b700fb service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:44.153448 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede04b4-da31-40f7-949e-8d38b0b700fb service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:01:44.155747 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fede04b4-da31-40f7-949e-8d38b0b700fb service glance] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:01:44.171007 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 737/1475] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:01:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:01:46.559669 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83716060-f8e7-43c9-98bc-d5494c5a9d78 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:46.560367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83716060-f8e7-43c9-98bc-d5494c5a9d78 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:46.560367 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83716060-f8e7-43c9-98bc-d5494c5a9d78 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:46.561598 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 739/1476] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 17:01:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:01:46.568189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad791bc-b864-478f-a3f5-900d19a5a046 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:46.569459 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad791bc-b864-478f-a3f5-900d19a5a046 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:46.569459 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad791bc-b864-478f-a3f5-900d19a5a046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:46.588987 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-1ad791bc-b864-478f-a3f5-900d19a5a046 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:01:46.590175 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-1ad791bc-b864-478f-a3f5-900d19a5a046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:01:46.599892 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 738/1477] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 17:01:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:01:46.635455 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83716060-f8e7-43c9-98bc-d5494c5a9d78 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:01:46.657208 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83716060-f8e7-43c9-98bc-d5494c5a9d78 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:01:46.665993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ab795-76e5-4ecd-b56b-5c79d04f8e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841c10778a1544ddb64ccc3d96fecf42', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:01:46.667095 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ab795-76e5-4ecd-b56b-5c79d04f8e9f admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:46.667465 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ab795-76e5-4ecd-b56b-5c79d04f8e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:46.667760 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ab795-76e5-4ecd-b56b-5c79d04f8e9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:46.668502 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ab795-76e5-4ecd-b56b-5c79d04f8e9f admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:01:46.670282 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: WARNING py.warnings [None req-3a7ab795-76e5-4ecd-b56b-5c79d04f8e9f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 17:01:46.670282 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: warnings.warn(msg) May 23 17:01:46.670282 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: May 23 17:01:46.671445 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ab795-76e5-4ecd-b56b-5c79d04f8e9f admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:01:46.691446 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 740/1478] 10.222.0.13 () {56 vars in 1108 bytes} [Mon May 23 17:01:46 2022] GET /identity/v3/projects => generated 2553 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 17:01:48.922040 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8721767-8415-4607-a09b-4a8420b3ffd5 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:48.922929 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8721767-8415-4607-a09b-4a8420b3ffd5 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:48.922929 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8721767-8415-4607-a09b-4a8420b3ffd5 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:48.925066 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 739/1479] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 17:01:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:01:48.933687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56786076-1aba-43c2-96e6-178d4f81ea78 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:48.934147 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56786076-1aba-43c2-96e6-178d4f81ea78 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:48.934298 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56786076-1aba-43c2-96e6-178d4f81ea78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:48.958190 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-56786076-1aba-43c2-96e6-178d4f81ea78 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:01:48.959599 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-56786076-1aba-43c2-96e6-178d4f81ea78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:01:48.969216 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 741/1480] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 17:01:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:01:49.014583 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8721767-8415-4607-a09b-4a8420b3ffd5 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:01:49.031262 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8721767-8415-4607-a09b-4a8420b3ffd5 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:01:49.037565 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2fb708-1c82-4ea4-934c-262ee22a9664 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:01:49.038208 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fb708-1c82-4ea4-934c-262ee22a9664 service nova] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:49.038493 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fb708-1c82-4ea4-934c-262ee22a9664 service nova] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:49.038766 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fb708-1c82-4ea4-934c-262ee22a9664 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:49.047314 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2fb708-1c82-4ea4-934c-262ee22a9664 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:01:49.048894 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2fb708-1c82-4ea4-934c-262ee22a9664 service nova] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:01:49.058464 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 740/1481] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:01:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:01:51.431253 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abf523-12cf-4220-9bcb-c8f495274af6 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:51.431715 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abf523-12cf-4220-9bcb-c8f495274af6 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:51.431823 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98abf523-12cf-4220-9bcb-c8f495274af6 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:51.433197 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 742/1482] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 17:01:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:01:51.438787 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0cbfb-58d3-4db5-951f-942f94d2d741 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:51.439266 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0cbfb-58d3-4db5-951f-942f94d2d741 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:51.439387 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d0cbfb-58d3-4db5-951f-942f94d2d741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:51.458479 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-97d0cbfb-58d3-4db5-951f-942f94d2d741 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:01:51.459731 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-97d0cbfb-58d3-4db5-951f-942f94d2d741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:01:51.468890 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 741/1483] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 17:01:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:01:51.504329 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98abf523-12cf-4220-9bcb-c8f495274af6 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:01:51.519139 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98abf523-12cf-4220-9bcb-c8f495274af6 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:01:51.524224 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87d2204-26ab-457a-9f17-987c4fe93b35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:01:51.524855 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d2204-26ab-457a-9f17-987c4fe93b35 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:51.525131 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d2204-26ab-457a-9f17-987c4fe93b35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:51.525398 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87d2204-26ab-457a-9f17-987c4fe93b35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:51.535597 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d2204-26ab-457a-9f17-987c4fe93b35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:01:51.537051 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87d2204-26ab-457a-9f17-987c4fe93b35 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:01:51.546890 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 743/1484] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:01:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:01:53.795586 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991e166-db55-48e1-84f4-a84862aed4d1 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:53.796429 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991e166-db55-48e1-84f4-a84862aed4d1 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:53.796429 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b991e166-db55-48e1-84f4-a84862aed4d1 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:53.798406 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 742/1485] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 17:01:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:01:53.806062 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eff38f-0d49-4d72-af52-7de4124b3045 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:53.806498 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eff38f-0d49-4d72-af52-7de4124b3045 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:53.806602 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eff38f-0d49-4d72-af52-7de4124b3045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:53.827973 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-57eff38f-0d49-4d72-af52-7de4124b3045 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:01:53.829152 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-57eff38f-0d49-4d72-af52-7de4124b3045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:01:53.838596 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 744/1486] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 17:01:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:01:53.875652 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991e166-db55-48e1-84f4-a84862aed4d1 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:01:53.887985 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b991e166-db55-48e1-84f4-a84862aed4d1 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:01:53.892480 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51149a8c-b737-4dcc-b81b-58dc4e9f098b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:01:53.893118 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51149a8c-b737-4dcc-b81b-58dc4e9f098b service neutron] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:53.893397 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51149a8c-b737-4dcc-b81b-58dc4e9f098b service neutron] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:53.893667 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51149a8c-b737-4dcc-b81b-58dc4e9f098b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:53.903248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51149a8c-b737-4dcc-b81b-58dc4e9f098b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:01:53.904685 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51149a8c-b737-4dcc-b81b-58dc4e9f098b service neutron] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:01:53.915085 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 743/1487] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:01:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:01:56.270138 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21adaa0-26b2-4ab9-933f-b3a32c032441 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:56.270759 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21adaa0-26b2-4ab9-933f-b3a32c032441 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:56.270759 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21adaa0-26b2-4ab9-933f-b3a32c032441 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:56.272430 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 745/1488] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 17:01:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:01:56.278122 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff1786-93fd-4aca-bb48-d917dee7aa9f None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:56.278597 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff1786-93fd-4aca-bb48-d917dee7aa9f None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:56.278701 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ff1786-93fd-4aca-bb48-d917dee7aa9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:56.304856 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-f4ff1786-93fd-4aca-bb48-d917dee7aa9f None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:01:56.306047 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-f4ff1786-93fd-4aca-bb48-d917dee7aa9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:01:56.315259 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 744/1489] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 17:01:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 17:01:56.358420 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21adaa0-26b2-4ab9-933f-b3a32c032441 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:01:56.377726 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21adaa0-26b2-4ab9-933f-b3a32c032441 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:01:56.384138 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7751845d-daa8-4eee-bca4-1cf7a9ae6645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:01:56.384790 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751845d-daa8-4eee-bca4-1cf7a9ae6645 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:56.385066 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751845d-daa8-4eee-bca4-1cf7a9ae6645 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:56.385324 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751845d-daa8-4eee-bca4-1cf7a9ae6645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:56.395726 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751845d-daa8-4eee-bca4-1cf7a9ae6645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:01:56.397179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7751845d-daa8-4eee-bca4-1cf7a9ae6645 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:01:56.407214 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 746/1490] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:01:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:01:58.824640 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077d169-9424-407f-aa00-62b4657732af None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:58.825528 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077d169-9424-407f-aa00-62b4657732af None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:58.825528 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6077d169-9424-407f-aa00-62b4657732af None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:58.827680 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 745/1491] 10.222.0.13 () {54 vars in 926 bytes} [Mon May 23 17:01:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:01:58.834511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55df3b-b04d-4bd9-8931-32f54fcfe000 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:58.835216 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55df3b-b04d-4bd9-8931-32f54fcfe000 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:58.835411 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c55df3b-b04d-4bd9-8931-32f54fcfe000 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:58.859657 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-3c55df3b-b04d-4bd9-8931-32f54fcfe000 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:01:58.860852 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-3c55df3b-b04d-4bd9-8931-32f54fcfe000 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:01:58.870003 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 747/1492] 10.222.0.13 () {58 vars in 1023 bytes} [Mon May 23 17:01:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:01:58.912506 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077d169-9424-407f-aa00-62b4657732af None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:01:58.925104 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6077d169-9424-407f-aa00-62b4657732af None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:01:58.929998 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25dd657f-a5b6-45a0-a43c-fdd504b0d245 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:01:58.930627 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd657f-a5b6-45a0-a43c-fdd504b0d245 service neutron] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:01:58.930903 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd657f-a5b6-45a0-a43c-fdd504b0d245 service neutron] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:01:58.931267 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dd657f-a5b6-45a0-a43c-fdd504b0d245 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:01:58.941546 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd657f-a5b6-45a0-a43c-fdd504b0d245 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:01:58.942943 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25dd657f-a5b6-45a0-a43c-fdd504b0d245 service neutron] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:01:58.952326 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 746/1493] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:01:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:02:08.854055 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=33 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 17:02:08.948186 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=33 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 17:03:45.643553 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3de5e-786d-412a-801b-7032d62c3dc5 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:03:45.644258 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3de5e-786d-412a-801b-7032d62c3dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:03:45.644258 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b3de5e-786d-412a-801b-7032d62c3dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:03:45.663338 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-52b3de5e-786d-412a-801b-7032d62c3dc5 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:03:45.664536 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-52b3de5e-786d-412a-801b-7032d62c3dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:03:45.673722 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 748/1494] 10.222.0.13 () {58 vars in 957 bytes} [Mon May 23 17:03:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:03:45.683306 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f131c7-4586-4c45-ae41-3533c9091a82 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:03:45.683826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f131c7-4586-4c45-ae41-3533c9091a82 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:03:45.683826 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f131c7-4586-4c45-ae41-3533c9091a82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:03:45.702576 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-a0f131c7-4586-4c45-ae41-3533c9091a82 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:03:45.703734 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-a0f131c7-4586-4c45-ae41-3533c9091a82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:03:45.712356 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 747/1495] 10.222.0.13 () {58 vars in 957 bytes} [Mon May 23 17:03:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:03:45.718687 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3de5e-786d-412a-801b-7032d62c3dc5 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:03:45.731312 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b3de5e-786d-412a-801b-7032d62c3dc5 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:03:45.739077 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc3a7f-4df7-4629-86d4-f1c14cd62345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841c10778a1544ddb64ccc3d96fecf42', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:03:45.739741 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc3a7f-4df7-4629-86d4-f1c14cd62345 admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:03:45.740024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc3a7f-4df7-4629-86d4-f1c14cd62345 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:03:45.740320 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68dc3a7f-4df7-4629-86d4-f1c14cd62345 admin admin] PATH_INFO: `/v3/domains` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:03:45.741191 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc3a7f-4df7-4629-86d4-f1c14cd62345 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80464) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 17:03:45.741519 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc3a7f-4df7-4629-86d4-f1c14cd62345 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:03:45.742561 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: WARNING py.warnings [None req-68dc3a7f-4df7-4629-86d4-f1c14cd62345 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 17:03:45.742561 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: warnings.warn(msg) May 23 17:03:45.742561 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: May 23 17:03:45.743480 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68dc3a7f-4df7-4629-86d4-f1c14cd62345 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:03:45.756879 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 749/1496] 10.222.0.13 () {58 vars in 1152 bytes} [Mon May 23 17:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 17:03:45.765428 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f131c7-4586-4c45-ae41-3533c9091a82 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:03:45.775601 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f131c7-4586-4c45-ae41-3533c9091a82 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:03:45.781879 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f2da8-a925-4408-9a45-3201d90d28d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841c10778a1544ddb64ccc3d96fecf42', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:03:45.782534 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f2da8-a925-4408-9a45-3201d90d28d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:03:45.782886 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f2da8-a925-4408-9a45-3201d90d28d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:03:45.783190 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f2da8-a925-4408-9a45-3201d90d28d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:03:45.783886 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f2da8-a925-4408-9a45-3201d90d28d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:03:45.785752 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: WARNING py.warnings [None req-784f2da8-a925-4408-9a45-3201d90d28d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 17:03:45.785752 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: warnings.warn(msg) May 23 17:03:45.785752 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: May 23 17:03:45.786970 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-784f2da8-a925-4408-9a45-3201d90d28d3 admin admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:03:45.829476 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 748/1497] 10.222.0.13 () {60 vars in 1152 bytes} [Mon May 23 17:03:45 2022] POST /identity/v3/projects => generated 379 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 17:03:45.833846 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc3a7f-4df7-4629-86d4-f1c14cd62345 admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:03:45.843985 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68dc3a7f-4df7-4629-86d4-f1c14cd62345 admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:03:45.903819 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c73cb4-8d15-42c2-b643-e15f13d015ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841c10778a1544ddb64ccc3d96fecf42', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:03:45.904487 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c73cb4-8d15-42c2-b643-e15f13d015ab admin admin] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:03:45.904764 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c73cb4-8d15-42c2-b643-e15f13d015ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:03:45.905011 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c73cb4-8d15-42c2-b643-e15f13d015ab admin admin] PATH_INFO: `/v3/users` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:03:45.905703 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c73cb4-8d15-42c2-b643-e15f13d015ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:03:45.906726 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: WARNING py.warnings [None req-e9c73cb4-8d15-42c2-b643-e15f13d015ab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 17:03:45.906726 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: warnings.warn(msg) May 23 17:03:45.906726 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: May 23 17:03:45.907957 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c73cb4-8d15-42c2-b643-e15f13d015ab admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:03:45.930132 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 750/1498] 10.222.0.13 () {60 vars in 1143 bytes} [Mon May 23 17:03:45 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 17:03:45.933609 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f2da8-a925-4408-9a45-3201d90d28d3 admin admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:03:45.943845 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f2da8-a925-4408-9a45-3201d90d28d3 admin admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:03:45.948403 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329febbe-b44f-4b9d-95ea-3f17ba29535a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841c10778a1544ddb64ccc3d96fecf42', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:03:45.949067 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329febbe-b44f-4b9d-95ea-3f17ba29535a admin admin] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:03:45.949346 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329febbe-b44f-4b9d-95ea-3f17ba29535a admin admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:03:45.949614 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329febbe-b44f-4b9d-95ea-3f17ba29535a admin admin] PATH_INFO: `/v3/roles` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:03:45.950336 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329febbe-b44f-4b9d-95ea-3f17ba29535a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:03:45.951368 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: WARNING py.warnings [None req-329febbe-b44f-4b9d-95ea-3f17ba29535a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 17:03:45.951368 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: warnings.warn(msg) May 23 17:03:45.951368 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: May 23 17:03:45.952240 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329febbe-b44f-4b9d-95ea-3f17ba29535a admin admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:03:45.964845 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 749/1499] 10.222.0.13 () {58 vars in 1121 bytes} [Mon May 23 17:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 17:03:45.969023 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c73cb4-8d15-42c2-b643-e15f13d015ab admin admin] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:03:45.979653 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c73cb4-8d15-42c2-b643-e15f13d015ab admin admin] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:03:45.985759 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e97cbbf-b6ef-4a8c-b6d3-a802bef150ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841c10778a1544ddb64ccc3d96fecf42', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:03:45.986585 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97cbbf-b6ef-4a8c-b6d3-a802bef150ec admin admin] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:03:45.986936 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97cbbf-b6ef-4a8c-b6d3-a802bef150ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:03:45.987230 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e97cbbf-b6ef-4a8c-b6d3-a802bef150ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:03:45.988062 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97cbbf-b6ef-4a8c-b6d3-a802bef150ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:03:45.989093 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: WARNING py.warnings [None req-2e97cbbf-b6ef-4a8c-b6d3-a802bef150ec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 17:03:45.989093 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: warnings.warn(msg) May 23 17:03:45.989093 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: May 23 17:03:45.989985 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e97cbbf-b6ef-4a8c-b6d3-a802bef150ec admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:03:46.004400 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 751/1500] 10.222.0.13 () {58 vars in 1121 bytes} [Mon May 23 17:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 17:03:46.009073 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329febbe-b44f-4b9d-95ea-3f17ba29535a admin admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:03:46.019736 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329febbe-b44f-4b9d-95ea-3f17ba29535a admin admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:03:46.024844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935e4004-3905-45cc-aa05-6ac6e3573f65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841c10778a1544ddb64ccc3d96fecf42', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:03:46.025731 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e4004-3905-45cc-aa05-6ac6e3573f65 admin admin] REQUEST_METHOD: `PUT` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:03:46.026016 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e4004-3905-45cc-aa05-6ac6e3573f65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:03:46.026291 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935e4004-3905-45cc-aa05-6ac6e3573f65 admin admin] PATH_INFO: `/v3/projects/e818cd1955854c8bb97aefbfa2d4df1f/users/b62a846df61d461a8b11bf85b35ed8fd/roles/72cc26a1a66c4fc183b39fad4cb38431` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:03:46.042286 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e4004-3905-45cc-aa05-6ac6e3573f65 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e818cd1955854c8bb97aefbfa2d4df1f, user_id=b62a846df61d461a8b11bf85b35ed8fd, role_id=72cc26a1a66c4fc183b39fad4cb38431)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:03:46.043360 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: WARNING py.warnings [None req-935e4004-3905-45cc-aa05-6ac6e3573f65 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 17:03:46.043360 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: warnings.warn(msg) May 23 17:03:46.043360 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: May 23 17:03:46.045310 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935e4004-3905-45cc-aa05-6ac6e3573f65 admin admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:03:46.063289 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 750/1501] 10.222.0.13 () {60 vars in 1482 bytes} [Mon May 23 17:03:46 2022] PUT /identity/v3/projects/e818cd1955854c8bb97aefbfa2d4df1f/users/b62a846df61d461a8b11bf85b35ed8fd/roles/72cc26a1a66c4fc183b39fad4cb38431 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 17:03:46.075081 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96379dd4-3469-4cff-8361-55576b773abd None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:03:46.075372 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96379dd4-3469-4cff-8361-55576b773abd None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:03:46.075646 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96379dd4-3469-4cff-8361-55576b773abd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:03:46.104188 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-96379dd4-3469-4cff-8361-55576b773abd None None] MFA Rules not processed for user `b62a846df61d461a8b11bf85b35ed8fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:03:46.105078 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-96379dd4-3469-4cff-8361-55576b773abd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:03:46.173062 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 752/1502] 10.222.0.13 () {58 vars in 957 bytes} [Mon May 23 17:03:46 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:03:46.186796 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0942e9-07e4-447e-b10f-7b37ccda61a3 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:03:46.187120 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0942e9-07e4-447e-b10f-7b37ccda61a3 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:03:46.187396 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0942e9-07e4-447e-b10f-7b37ccda61a3 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:03:46.188717 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 751/1503] 10.222.0.13 () {54 vars in 963 bytes} [Mon May 23 17:03:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:03:46.193893 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8092f9-18de-41a4-8b9e-447a33ad3350 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:03:46.194396 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8092f9-18de-41a4-8b9e-447a33ad3350 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:03:46.194676 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8092f9-18de-41a4-8b9e-447a33ad3350 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:03:46.213396 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-9b8092f9-18de-41a4-8b9e-447a33ad3350 None None] MFA Rules not processed for user `22e45ab9d34c4fcbaf93be339257eca8`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:03:46.214610 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-9b8092f9-18de-41a4-8b9e-447a33ad3350 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:03:46.269734 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 17:03:46.278154 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 753/1504] 10.222.0.13 () {58 vars in 1060 bytes} [Mon May 23 17:03:46 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:03:46.286588 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0942e9-07e4-447e-b10f-7b37ccda61a3 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:03:46.297834 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0942e9-07e4-447e-b10f-7b37ccda61a3 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:03:46.302646 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70677d5f-a9f9-4c55-aac7-0203e72f3429 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22e45ab9d34c4fcbaf93be339257eca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:03:46.303613 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70677d5f-a9f9-4c55-aac7-0203e72f3429 service cinder] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:03:46.304100 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70677d5f-a9f9-4c55-aac7-0203e72f3429 service cinder] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:03:46.304422 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70677d5f-a9f9-4c55-aac7-0203e72f3429 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:03:46.316675 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70677d5f-a9f9-4c55-aac7-0203e72f3429 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:03:46.318782 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70677d5f-a9f9-4c55-aac7-0203e72f3429 service cinder] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:03:46.328554 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 752/1505] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:03:46 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:03:46.375017 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d8571-5424-46dc-9553-da790a27a3f4 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:03:46.375327 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d8571-5424-46dc-9553-da790a27a3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:03:46.375595 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058d8571-5424-46dc-9553-da790a27a3f4 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:03:46.377470 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 754/1506] 10.222.0.13 () {52 vars in 829 bytes} [Mon May 23 17:03:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:03:46.382054 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70677d5f-a9f9-4c55-aac7-0203e72f3429 service cinder] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:03:46.453060 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70677d5f-a9f9-4c55-aac7-0203e72f3429 service cinder] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:03:46.459120 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9550eba-c388-4483-b832-74e6dca3e725 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841c10778a1544ddb64ccc3d96fecf42', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:03:46.460200 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9550eba-c388-4483-b832-74e6dca3e725 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:03:46.460474 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9550eba-c388-4483-b832-74e6dca3e725 admin admin] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:03:46.460750 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9550eba-c388-4483-b832-74e6dca3e725 admin admin] PATH_INFO: `/v3/users/b62a846df61d461a8b11bf85b35ed8fd` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:03:46.461990 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9550eba-c388-4483-b832-74e6dca3e725 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b62a846df61d461a8b11bf85b35ed8fd)` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:03:46.463024 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: WARNING py.warnings [None req-e9550eba-c388-4483-b832-74e6dca3e725 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 17:03:46.463024 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: warnings.warn(msg) May 23 17:03:46.463024 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: May 23 17:03:46.464241 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9550eba-c388-4483-b832-74e6dca3e725 admin admin] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:03:46.540113 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [None req-e9550eba-c388-4483-b832-74e6dca3e725 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b62a846df61d461a8b11bf85b35ed8fd'} {{(pid=80465) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 17:03:46.552144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [None req-e9550eba-c388-4483-b832-74e6dca3e725 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b62a846df61d461a8b11bf85b35ed8fd'} {{(pid=80465) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 17:03:46.564616 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [None req-e9550eba-c388-4483-b832-74e6dca3e725 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b62a846df61d461a8b11bf85b35ed8fd'} {{(pid=80465) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 17:03:46.581471 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 17:03:46.584097 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [None req-e9550eba-c388-4483-b832-74e6dca3e725 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b62a846df61d461a8b11bf85b35ed8fd'} {{(pid=80465) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 17:03:46.585068 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.notifications [None req-e9550eba-c388-4483-b832-74e6dca3e725 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b62a846df61d461a8b11bf85b35ed8fd'} {{(pid=80465) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 17:03:46.591938 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 753/1507] 10.222.0.13 () {58 vars in 1223 bytes} [Mon May 23 17:03:46 2022] DELETE /identity/v3/users/b62a846df61d461a8b11bf85b35ed8fd => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 17:03:46.596415 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c57d6a-1db9-40ad-b8fa-7fa2cc447dd1 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:03:46.596781 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c57d6a-1db9-40ad-b8fa-7fa2cc447dd1 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:03:46.597380 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c57d6a-1db9-40ad-b8fa-7fa2cc447dd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:03:46.615318 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-57c57d6a-1db9-40ad-b8fa-7fa2cc447dd1 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:03:46.616511 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-57c57d6a-1db9-40ad-b8fa-7fa2cc447dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:03:46.678559 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 755/1508] 10.222.0.13 () {58 vars in 957 bytes} [Mon May 23 17:03:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:03:46.688912 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9550eba-c388-4483-b832-74e6dca3e725 admin admin] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:03:46.690412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-e9550eba-c388-4483-b832-74e6dca3e725 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:03:46.763033 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9550eba-c388-4483-b832-74e6dca3e725 admin admin] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:03:46.767186 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99dbe93-6871-4533-95a2-0347ae2292b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:03:46.767844 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99dbe93-6871-4533-95a2-0347ae2292b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:03:46.768153 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99dbe93-6871-4533-95a2-0347ae2292b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:03:46.768423 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99dbe93-6871-4533-95a2-0347ae2292b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:03:46.779258 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99dbe93-6871-4533-95a2-0347ae2292b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:03:46.780718 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b99dbe93-6871-4533-95a2-0347ae2292b6 service neutron] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:03:46.790865 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 754/1509] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:03:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 17:03:47.236142 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c57d6a-1db9-40ad-b8fa-7fa2cc447dd1 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:03:47.238503 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-57c57d6a-1db9-40ad-b8fa-7fa2cc447dd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:03:47.250483 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c57d6a-1db9-40ad-b8fa-7fa2cc447dd1 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:03:47.256023 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc44c5c-5815-48bc-95f6-e34158950723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd1134eaf564846f5990bd2b2934b7db7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '841c10778a1544ddb64ccc3d96fecf42', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:03:47.256878 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc44c5c-5815-48bc-95f6-e34158950723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:03:47.257153 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc44c5c-5815-48bc-95f6-e34158950723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:03:47.257419 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc44c5c-5815-48bc-95f6-e34158950723 admin admin] PATH_INFO: `/v3/projects/e818cd1955854c8bb97aefbfa2d4df1f` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:03:47.258712 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc44c5c-5815-48bc-95f6-e34158950723 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e818cd1955854c8bb97aefbfa2d4df1f)` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:03:47.259805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: WARNING py.warnings [None req-ebc44c5c-5815-48bc-95f6-e34158950723 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 17:03:47.259805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: warnings.warn(msg) May 23 17:03:47.259805 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: May 23 17:03:47.261008 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc44c5c-5815-48bc-95f6-e34158950723 admin admin] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:03:47.330551 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-ebc44c5c-5815-48bc-95f6-e34158950723 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e818cd1955854c8bb97aefbfa2d4df1f', 'request_id': 'req-ebc44c5c-5815-48bc-95f6-e34158950723'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 17:03:47.336757 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-ebc44c5c-5815-48bc-95f6-e34158950723 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e818cd1955854c8bb97aefbfa2d4df1f', 'request_id': 'req-ebc44c5c-5815-48bc-95f6-e34158950723'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 17:03:47.348275 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-ebc44c5c-5815-48bc-95f6-e34158950723 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e818cd1955854c8bb97aefbfa2d4df1f', 'request_id': 'req-ebc44c5c-5815-48bc-95f6-e34158950723'} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 17:03:47.367615 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-ebc44c5c-5815-48bc-95f6-e34158950723 admin admin] The token cache is being invalidate because project e818cd1955854c8bb97aefbfa2d4df1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80464) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 17:03:47.367917 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-ebc44c5c-5815-48bc-95f6-e34158950723 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 17:03:47.368718 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.notifications [None req-ebc44c5c-5815-48bc-95f6-e34158950723 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80464) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 17:03:47.370350 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 756/1510] 10.222.0.13 () {58 vars in 1232 bytes} [Mon May 23 17:03:47 2022] DELETE /identity/v3/projects/e818cd1955854c8bb97aefbfa2d4df1f => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 17:03:56.786468 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 17:03:57.364156 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 17:05:29.317201 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeaa2dd-376d-44e2-ae30-9459c43c83f5 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:29.318114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeaa2dd-376d-44e2-ae30-9459c43c83f5 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:29.318114 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aeaa2dd-376d-44e2-ae30-9459c43c83f5 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:29.320256 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 755/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 17:05:29 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:05:29.326113 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87592cf-2271-49a9-a76a-2e039e5aee81 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:29.326574 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87592cf-2271-49a9-a76a-2e039e5aee81 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:29.326753 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87592cf-2271-49a9-a76a-2e039e5aee81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:29.354024 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-c87592cf-2271-49a9-a76a-2e039e5aee81 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:05:29.355277 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-c87592cf-2271-49a9-a76a-2e039e5aee81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:05:29.420672 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 757/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 17:05:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:05:29.595408 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeaa2dd-376d-44e2-ae30-9459c43c83f5 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:05:29.598460 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-7aeaa2dd-376d-44e2-ae30-9459c43c83f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:05:29.813748 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aeaa2dd-376d-44e2-ae30-9459c43c83f5 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:05:29.820133 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19969026-bf37-4840-bf6d-b5a500b1504a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:05:29.820722 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19969026-bf37-4840-bf6d-b5a500b1504a service nova] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:29.821000 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19969026-bf37-4840-bf6d-b5a500b1504a service nova] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:29.821265 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19969026-bf37-4840-bf6d-b5a500b1504a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:29.830451 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19969026-bf37-4840-bf6d-b5a500b1504a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:05:29.832133 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19969026-bf37-4840-bf6d-b5a500b1504a service nova] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:05:29.841508 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 756/1513] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:05:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:05:30.219753 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12681227-03a5-4a68-81f6-0cd8b7a8ba09 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:30.220184 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12681227-03a5-4a68-81f6-0cd8b7a8ba09 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:30.220462 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12681227-03a5-4a68-81f6-0cd8b7a8ba09 None None] PATH_INFO: `/v3` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:30.221891 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 758/1514] 10.222.0.11 () {54 vars in 925 bytes} [Mon May 23 17:05:30 2022] GET /identity/v3 => generated 255 bytes in 4 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 17:05:30.228701 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11f49d-92da-4bd2-a1ce-8c1acab6fbf3 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:30.229028 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11f49d-92da-4bd2-a1ce-8c1acab6fbf3 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:30.229308 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db11f49d-92da-4bd2-a1ce-8c1acab6fbf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:30.247678 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-db11f49d-92da-4bd2-a1ce-8c1acab6fbf3 None None] MFA Rules not processed for user `f181ebdfcd544b54af7e6f9c118fe252`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:05:30.248897 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-db11f49d-92da-4bd2-a1ce-8c1acab6fbf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:05:30.314301 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 757/1515] 10.222.0.11 () {58 vars in 1015 bytes} [Mon May 23 17:05:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:05:30.744362 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d54b59-426d-4f41-a088-07d625f320e1 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:30.745225 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d54b59-426d-4f41-a088-07d625f320e1 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:30.745225 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d54b59-426d-4f41-a088-07d625f320e1 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:30.746984 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 759/1516] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 17:05:30 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:05:30.755731 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0748a9-02bd-4702-bd73-22e660767e1a None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:30.756177 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0748a9-02bd-4702-bd73-22e660767e1a None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:30.756336 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0748a9-02bd-4702-bd73-22e660767e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:30.778731 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-3a0748a9-02bd-4702-bd73-22e660767e1a None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:05:30.779961 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-3a0748a9-02bd-4702-bd73-22e660767e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:05:30.788003 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 758/1517] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 17:05:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:05:30.940225 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d54b59-426d-4f41-a088-07d625f320e1 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:05:30.965682 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d54b59-426d-4f41-a088-07d625f320e1 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:05:30.973542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2d187f-ab21-4357-918c-b1d74217105f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:05:30.974179 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d187f-ab21-4357-918c-b1d74217105f service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:30.974462 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d187f-ab21-4357-918c-b1d74217105f service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:30.974734 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2d187f-ab21-4357-918c-b1d74217105f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:30.983665 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d187f-ab21-4357-918c-b1d74217105f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:05:30.985315 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2d187f-ab21-4357-918c-b1d74217105f service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:05:30.995421 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 760/1518] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:05:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:05:34.179947 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e753e6-6833-4b7b-9b78-b6da2e2354c1 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:34.180535 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e753e6-6833-4b7b-9b78-b6da2e2354c1 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:34.180535 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e753e6-6833-4b7b-9b78-b6da2e2354c1 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:34.182000 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 759/1519] 10.222.0.13 () {54 vars in 960 bytes} [Mon May 23 17:05:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:05:34.187088 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c322ac7-6209-4717-9c15-10941f2c05dc None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:34.188061 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c322ac7-6209-4717-9c15-10941f2c05dc None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:34.188410 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c322ac7-6209-4717-9c15-10941f2c05dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:34.211424 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-7c322ac7-6209-4717-9c15-10941f2c05dc None None] MFA Rules not processed for user `f181ebdfcd544b54af7e6f9c118fe252`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:05:34.212736 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-7c322ac7-6209-4717-9c15-10941f2c05dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:05:34.223279 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 761/1520] 10.222.0.13 () {58 vars in 1057 bytes} [Mon May 23 17:05:34 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:05:34.231406 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e753e6-6833-4b7b-9b78-b6da2e2354c1 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:05:34.245473 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e753e6-6833-4b7b-9b78-b6da2e2354c1 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:05:34.250888 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12ad4960-775f-4174-86f6-401ab47572bd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f181ebdfcd544b54af7e6f9c118fe252', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:05:34.251568 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad4960-775f-4174-86f6-401ab47572bd service placement] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:34.251858 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad4960-775f-4174-86f6-401ab47572bd service placement] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:34.252133 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12ad4960-775f-4174-86f6-401ab47572bd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:34.261627 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad4960-775f-4174-86f6-401ab47572bd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:05:34.263287 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12ad4960-775f-4174-86f6-401ab47572bd service placement] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:05:34.273368 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 760/1521] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:05:34 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:05:42.288557 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15707162-62db-483d-8d0c-09191be98514 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:42.289413 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15707162-62db-483d-8d0c-09191be98514 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:42.289413 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15707162-62db-483d-8d0c-09191be98514 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:42.291621 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 762/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 17:05:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:05:42.299635 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0731cc1-b340-4e2d-a82a-518c2f68f922 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:42.300058 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0731cc1-b340-4e2d-a82a-518c2f68f922 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:42.300204 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0731cc1-b340-4e2d-a82a-518c2f68f922 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:42.324102 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-b0731cc1-b340-4e2d-a82a-518c2f68f922 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:05:42.325306 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-b0731cc1-b340-4e2d-a82a-518c2f68f922 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:05:42.334507 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 761/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 17:05:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:05:42.487917 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15707162-62db-483d-8d0c-09191be98514 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:05:42.503044 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15707162-62db-483d-8d0c-09191be98514 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:05:42.508928 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d6a89-811b-4982-8332-b6dbd16f8a78 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:05:42.509582 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d6a89-811b-4982-8332-b6dbd16f8a78 service nova] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:42.509863 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d6a89-811b-4982-8332-b6dbd16f8a78 service nova] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:42.510130 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d6a89-811b-4982-8332-b6dbd16f8a78 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:42.518837 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59d6a89-811b-4982-8332-b6dbd16f8a78 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:05:42.520476 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59d6a89-811b-4982-8332-b6dbd16f8a78 service nova] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:05:42.529885 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 763/1524] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:05:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:05:43.387248 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abceea9b-2432-4fed-a4e5-1e711f131031 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:43.388213 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abceea9b-2432-4fed-a4e5-1e711f131031 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:43.388213 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abceea9b-2432-4fed-a4e5-1e711f131031 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:43.390259 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 762/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 17:05:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:05:43.396360 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da3d89-869d-4029-a2d8-b98960383fc0 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:43.396816 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da3d89-869d-4029-a2d8-b98960383fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:43.396927 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da3d89-869d-4029-a2d8-b98960383fc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:43.423757 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-01da3d89-869d-4029-a2d8-b98960383fc0 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:05:43.424945 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-01da3d89-869d-4029-a2d8-b98960383fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:05:43.433922 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 764/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 17:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:05:43.589477 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abceea9b-2432-4fed-a4e5-1e711f131031 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:05:43.606681 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abceea9b-2432-4fed-a4e5-1e711f131031 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:05:43.612734 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994112f6-0945-4b8b-846d-b107a258b988 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '225c7b385f12445fab86b3b5606b1da1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'service', 'member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:05:43.613375 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994112f6-0945-4b8b-846d-b107a258b988 service nova] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:43.613659 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994112f6-0945-4b8b-846d-b107a258b988 service nova] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:43.613934 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994112f6-0945-4b8b-846d-b107a258b988 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:43.622348 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994112f6-0945-4b8b-846d-b107a258b988 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:05:43.623946 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994112f6-0945-4b8b-846d-b107a258b988 service nova] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:05:43.633766 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 763/1527] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:05:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:05:47.978182 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effd712-4cd6-4fdd-87d5-8e8cab677515 None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:47.978825 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effd712-4cd6-4fdd-87d5-8e8cab677515 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:47.978825 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8effd712-4cd6-4fdd-87d5-8e8cab677515 None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:47.980127 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 765/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 17:05:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:05:47.989144 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93976018-e7f3-4217-acb5-988848e8c4d8 None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:47.989850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93976018-e7f3-4217-acb5-988848e8c4d8 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:47.989850 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93976018-e7f3-4217-acb5-988848e8c4d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:48.014326 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-93976018-e7f3-4217-acb5-988848e8c4d8 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:05:48.016128 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-93976018-e7f3-4217-acb5-988848e8c4d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:05:48.025530 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 764/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 17:05:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:05:48.061901 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effd712-4cd6-4fdd-87d5-8e8cab677515 None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:05:48.064404 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-8effd712-4cd6-4fdd-87d5-8e8cab677515 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:05:48.128313 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8effd712-4cd6-4fdd-87d5-8e8cab677515 None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:05:48.132846 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e33b83-8264-4d21-a7fd-7691bcf59323 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:05:48.133484 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e33b83-8264-4d21-a7fd-7691bcf59323 service neutron] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:48.133763 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e33b83-8264-4d21-a7fd-7691bcf59323 service neutron] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:48.134029 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e33b83-8264-4d21-a7fd-7691bcf59323 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:48.144201 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e33b83-8264-4d21-a7fd-7691bcf59323 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:05:48.145648 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e33b83-8264-4d21-a7fd-7691bcf59323 service neutron] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:05:48.155212 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 766/1530] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:05:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:05:50.361071 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38240f3-ce66-4d6e-bd40-840e68fd9008 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:50.361982 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38240f3-ce66-4d6e-bd40-840e68fd9008 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:50.361982 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c38240f3-ce66-4d6e-bd40-840e68fd9008 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:50.364147 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 765/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 17:05:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:05:50.370542 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2afdb-567b-4254-b3b7-16ff6c692be6 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:50.371331 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2afdb-567b-4254-b3b7-16ff6c692be6 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:50.371477 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe2afdb-567b-4254-b3b7-16ff6c692be6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:50.392363 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-dbe2afdb-567b-4254-b3b7-16ff6c692be6 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:05:50.393552 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-dbe2afdb-567b-4254-b3b7-16ff6c692be6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:05:50.402831 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 767/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 17:05:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:05:50.439505 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38240f3-ce66-4d6e-bd40-840e68fd9008 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:05:50.441412 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-c38240f3-ce66-4d6e-bd40-840e68fd9008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:05:50.456354 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c38240f3-ce66-4d6e-bd40-840e68fd9008 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:05:50.460560 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c927f020-75cf-4f82-91fc-99853fd98c0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fd070e7581f04f4abcbb8b6bc231a8c1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:05:50.461189 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927f020-75cf-4f82-91fc-99853fd98c0f service neutron] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:50.461451 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927f020-75cf-4f82-91fc-99853fd98c0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:50.461746 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c927f020-75cf-4f82-91fc-99853fd98c0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:50.471859 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c927f020-75cf-4f82-91fc-99853fd98c0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:05:50.473278 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c927f020-75cf-4f82-91fc-99853fd98c0f service neutron] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:05:50.482965 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 766/1533] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:05:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:05:54.448213 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0637b1-0fae-492f-8ed1-d9ac9b94119a None None] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:54.449085 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0637b1-0fae-492f-8ed1-d9ac9b94119a None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:54.449085 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0637b1-0fae-492f-8ed1-d9ac9b94119a None None] PATH_INFO: `/` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:54.451328 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 768/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 17:05:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:05:54.456972 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29ae2a-656f-4c58-aac8-12cfa1d7f42e None None] REQUEST_METHOD: `POST` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:54.457421 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29ae2a-656f-4c58-aac8-12cfa1d7f42e None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:54.457529 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e29ae2a-656f-4c58-aac8-12cfa1d7f42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:54.477671 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.auth.core [None req-2e29ae2a-656f-4c58-aac8-12cfa1d7f42e None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80465) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:05:54.479633 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-2e29ae2a-656f-4c58-aac8-12cfa1d7f42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:05:54.489861 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 767/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 17:05:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:05:54.767673 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0637b1-0fae-492f-8ed1-d9ac9b94119a None None] Authenticating user token {{(pid=80464) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:05:54.769664 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-bd0637b1-0fae-492f-8ed1-d9ac9b94119a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:05:54.843658 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0637b1-0fae-492f-8ed1-d9ac9b94119a None None] Validating token access rules against request {{(pid=80464) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:05:54.848334 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b46b952-d6dd-4f83-be29-6f1fa0c6b77e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22e45ab9d34c4fcbaf93be339257eca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80464) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:05:54.848976 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46b952-d6dd-4f83-be29-6f1fa0c6b77e service cinder] REQUEST_METHOD: `GET` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:54.849256 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46b952-d6dd-4f83-be29-6f1fa0c6b77e service cinder] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:54.849529 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b46b952-d6dd-4f83-be29-6f1fa0c6b77e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:54.859664 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b46b952-d6dd-4f83-be29-6f1fa0c6b77e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:05:54.861141 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b46b952-d6dd-4f83-be29-6f1fa0c6b77e service cinder] RBAC: Authorization granted {{(pid=80464) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:05:54.866245 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=100 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 17:05:54.871755 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 769/1536] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:05:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:05:57.089120 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddcd27-bceb-4198-8d9e-b182a62ccd50 None None] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:57.089968 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddcd27-bceb-4198-8d9e-b182a62ccd50 None None] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:57.089968 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ddcd27-bceb-4198-8d9e-b182a62ccd50 None None] PATH_INFO: `/` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:57.092003 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 768/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 17:05:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 17:05:57.097520 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa93f5d-631c-4843-afe8-7fbfef6c6ed3 None None] REQUEST_METHOD: `POST` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:57.097993 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa93f5d-631c-4843-afe8-7fbfef6c6ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:57.098086 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa93f5d-631c-4843-afe8-7fbfef6c6ed3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80464) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:57.117753 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.auth.core [None req-efa93f5d-631c-4843-afe8-7fbfef6c6ed3 None None] MFA Rules not processed for user `d1134eaf564846f5990bd2b2934b7db7`. Rule list: `[]` (Enabled: `True`). {{(pid=80464) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 17:05:57.118961 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG keystone.common.fernet_utils [None req-efa93f5d-631c-4843-afe8-7fbfef6c6ed3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80464) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:05:57.129011 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: [pid: 80464|app: 0|req: 770/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 17:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 17:05:57.402549 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ddcd27-bceb-4198-8d9e-b182a62ccd50 None None] Authenticating user token {{(pid=80465) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 17:05:57.405616 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.fernet_utils [None req-73ddcd27-bceb-4198-8d9e-b182a62ccd50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80465) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 17:05:57.414157 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=94 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 17:05:57.421206 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ddcd27-bceb-4198-8d9e-b182a62ccd50 None None] Validating token access rules against request {{(pid=80465) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 17:05:57.425572 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80556ec3-9200-430a-a1c7-cf0c15729cbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '22e45ab9d34c4fcbaf93be339257eca8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0595667fb2e04a3ca4716f669d6698b0', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80465) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 17:05:57.426230 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80556ec3-9200-430a-a1c7-cf0c15729cbc service cinder] REQUEST_METHOD: `GET` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 17:05:57.426510 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80556ec3-9200-430a-a1c7-cf0c15729cbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 17:05:57.426784 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80556ec3-9200-430a-a1c7-cf0c15729cbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80465) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 17:05:57.436445 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80556ec3-9200-430a-a1c7-cf0c15729cbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 17:05:57.437909 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80556ec3-9200-430a-a1c7-cf0c15729cbc service cinder] RBAC: Authorization granted {{(pid=80465) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 17:05:57.447519 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: [pid: 80465|app: 0|req: 769/1539] 10.222.0.13 () {60 vars in 1365 bytes} [Mon May 23 17:05:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 17:06:07.112441 dv-cinder-842541-2-iscsi devstack@keystone.service[80464]: DEBUG dbcounter [-] [80464] Writing DB stats keystone:SELECT=3 {{(pid=80464) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 17:06:07.443601 dv-cinder-842541-2-iscsi devstack@keystone.service[80465]: DEBUG dbcounter [-] [80465] Writing DB stats keystone:SELECT=5 {{(pid=80465) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}